In my last post I wrote about how to Debug Custom .Net Activities in Azure Data Factory locally. This fixes one of the biggest issues in Azure Data Factory at the moment for developers. The next bigger problem that you will run into is when it comes to deploying your Azure Data Factory project. At the moment, you can only do it manually from Visual Studio which, for bigger projects, can take quite some time. So I extended and advanced the code from my CustomActivityDebugger. Well, actually I rewrote some major parts of it and moved it into a new GitHub repository: Azure.DataFactory.LocalEnvironment
The new code base now includes the functionality to export an existing ADF project to an ARM template which can then be deployed very easily using Azure standard deployment mechanisms.
So basically, these are the changes and new Features that I made:
Export as ARM template:
- Export all ADF objects and properties
- Support for configurations
- obey dependencies between ADF objects
- parameterized Data Factory name
- automatic upload of ADF dependencies (e.g. custom activities)
- specify the region where ADF should be deployed (ADF is not available in all regions yet!)
Custom Activity Debugger:
- simplified usability – just select the pipeline, activity and set the slice-dates
- Support for configurations
- no need to add any namespaces
- no need to add any references
- write activity log to console output
General:
- Load from the ADF Project file (.dfproj) instead of a whole folder
- implemented as Assembly
- can be used in a Console Application for automation
- will be published via NuGet in the future! (coming soon)
Everything else is described in the Git-Repository itself!
Hope you enjoy it!