Fault tolerance in Copy Activity by skipping incom

问题:

I use Azure Fucntion with Azure SDK and Azure Data Factory, is there any way to get value of skippedRowCount of Activity Window when applying "log the incompatible rows" in Copy Activity (Source: Blob Storage, Sink: SQL Data Warehouse)?

回答1:

From the documentation here : Monitor skipped rows, looks like a log file is automatically generated when log incompatible rows is enabled at https://[your-blob-account].blob.core.windows.net/[path-if-configured]/[copy-activity-run-id]/[auto-generated-GUID].csv. You can set up a blob trigger on the generated log file. Here is a sample function.json

"disabled": false,
"bindings": [
    {
        "name": "myBlob",
        "type": "blobTrigger",
        "direction": "in",
        "path": "[path-if-configured]/[copy-activity-run-id]",
        "connection":"MyStorageAccountAppSetting"
    }
]

Note: you could also use blob name patterns to specify path in the binding.

In the function code, you can easily access the contents of the log file to get the skippedRowCount