Tuesday, October 23, 2018

Copy Data from Azure SQL to Azure Data Lake via Azure Data Factory

First create app Registration and give access to Azure Data Lake

Now create a new app registration

Provide details for App registration

Copy the Application ID and then Generate a key
App Id - 4f0fcf16-e148-411e-8b91-92c600d83c9e


Once Key and AppId is created go back to Azure Data lake to provide permission

Go to Access and select the App Registration we created

Then Select permission for Read, Write and Execute

Now this new App Registration will have access for Azure Data Lake.

Now in Azure Data Factory  first go to Copy Data Wizard.

Now Select Source, which is SQL Server existing Connection

Create a new Azure Data Lake linked service for Destination

Provide the details of Azure Data Lake Store

Also Provide the Service Principal key to Access

Click on Test Connection - "Connection Successful" means connected.

Select Folder for Data Lake Store

Go Next  with default options

Summery before executing the pipeline.

Now you will see if Copy pipeline execution is Completed.

you can also see in Monitor tab for the Pipeline runs successfully.

Now go back to Data Lake Store  and click Data Explorer, you will see a new file created there for ADFDemo

Once you open this ADF Demo File you will see the comma separated values.

This is how we can copy data from SQL Server to Azure Data lake via Azure Data factory.

No comments:

Post a Comment