As you probably know from my previous posts, my colleagues at paiqo.com and I are constantly working to improve our VSCode extension for Databricks. Almost every month we silently release a new version to the VSCode gallery so you get the latest features. However, as this is a special release, I am also writing a dedicated blog post for it
We are very happy to finally announce the first official release of our VSCode extension for Databricks!
Probably the biggest and most awaited new feature is the ability to interactively execute your local Spark/SQL/Scala/R code against one of your running Databricks clusters and get the results back in VSCode! At least every 3rd issue that was opened in our Github Repo was requesting this feature and now it is finally here. It integrates seamlessly into VSCode notebooks by automatically creating a new kernel for every active Databricks cluster in your workspace. So just open one of your notebook, select the Databricks cluster kernel and start working!
Integrating it natively with VSCode notebooks brings some very nice features out of the box:
- leverages all other extensions that work with Python or notebooks:
syntax highlighting, auto-completion, colored indention levels (e.g using indent-rainbow) - use custom renderers to visualize your tables (e.g. vscode-data-tables)
In addition to all the features already mentioned above, it also solves some issues that you may had with other solutions like databricks-connect (which is also deprecated by now). The following things also work with the new Databricks VSCode notebook kernel:
- full access to the DBFS including mountpoints
- most dbutils functions like dbutils.secrets
- magics like %pip, %md and all language-specific magics
However, besides all those great new features and capabilities, there are also some things that still need improvement. Here is a list of known issues:
- widgets are currently not supported – that’s our highest priority at the moment
- features from files-in-repos are not yet supported
- only works with .ipynb files
- the notebook sometimes opens twice
- found any other issues? report them at at issues-page
All further details can be found in the README.md of the Github repository.
Let us know what you think of and provide some feedback! If you already had an older version of the extension installed, it will update automatically. If you have not used it yet, just install it directly via the VSCode extensions or download it from the VSCode gallery or our GitHub repository:
VS Code gallery: paiqo.Databricks-VSCode
Github repository: Databricks-VSCode