You are attempting a Delta Merge with automatic schema evolution, but it fails with a
Delta Merge: cannot resolve 'field' due to data type mismatch error message.
This can happen if you have made changes to the nested column fields.
For example, assume we have a column called
Address with the fields
city nested inside.
Attempting to add an additional field, or remove a field, causes any upcoming insert or update transaction on the table to fail, even if
mergeSchema is true for the transaction.
This behavior is by design.
The Delta automatic schema evolution feature only supports top level columns. Nested fields are not supported.
Please review the Delta Lake Automatic schema evolution documentation for more information.