Once you've installed, open a new command prompt and run the following command:
Once you've installed, open a new terminal and run the following command:
Command prompt
dotnet
If the installation succeeded, you should see an output similar to the following:
Command prompt
Usage: dotnet [options]Usage: dotnet [path-to-application]Options:-h|--help Display help.--info Display .NET information.--list-sdks Display the installed SDKs.--list-runtimes Display the installed runtimes.path-to-application:The path to an application .dll file to execute.
If everything looks good, select the Continue button below to go to the next step.
Got an error?
If you receive a 'dotnet' is not recognized as an internal or external command error, make sure you opened a new command prompt. If you can't resolve the issue, use the I ran into an issue button to get help fixing the problem.
If you receive a dotnet: command not found error, make sure you opened a new terminal window. If you can't resolve the issue, use the I ran into an issue button to get help fixing the problem.
Install Prerequisites
.NET for Apache Spark runs on the 64-bit version of the Windows operating system and assumes the same from this point onwards.
Install 7-zip
Apache Spark is downloaded as a compressed .tgz file. You'll need 7-zip to extract the file.
If you already have an alternative extraction program installed, you can use that instead.
In the first table on the page, select the 64-bit x64 download.
When the download completes, run the installer.
Install Java
Apache Spark requires the Java SE Development Kit (JDK) 8 or 11. This tutorial uses version 8, but you can use version 11 if you already have that installed.
Once you've installed everything and set your environment variables, open a new command prompt or terminal and run the following command:
Command prompt
spark-submit --version
If the command runs and prints version information, you can move to the next step. If you receive a 'spark-submit' is not recognized as an internal or external command error, make sure you opened a new command prompt.
Extract Apache Spark
In your terminal, move to the folder that contains the file you just downloaded then run the following command:
In your command prompt, run the following command to create your app:
In your terminal, run the following command to create your app:
Command prompt
dotnet new console -f netcoreapp3.1 -o mySparkApp
Then, navigate to the new directory created by the previous command:
Command prompt
cd mySparkApp
The dotnet command creates a new application of type console for you. The -o parameter creates a directory named mySparkApp where your app is stored and populates it with the required files. The cd mySparkApp command puts you into the newly created app directory.
Install the NuGet package
To use .NET for Apache Spark in an app, you need to add the Microsoft.Spark package to your project. In your command prompt, run the following command:
To use .NET for Apache Spark in an app, you need to add the Microsoft.Spark package to your project. In your terminal, run the following command:
You may encounter an error that Spark failed to delete a jar file from a temporary directory. This is a known error and won't affect the output of your app.
Next steps
Congratulations, you've built and run your first .NET for Apache Spark app!
Keep learning
If you want to keep learning more about .NET for Apache Spark, visit the following official documentation:
