.NET for Apache Spark™ Tutorial - Get started in 10 minutes

Run your app

Run the following command to build your application after navigating to the directory containing the application:

Command prompt
dotnet build

Run the following command to submit your application to run on Apache Spark (Note: please make sure input.txt exists in your mySparkApp directory):

Command prompt
spark-submit --class org.apache.spark.deploy.dotnet.DotnetRunner --master local bin\Debug\netcoreapp3.1\microsoft-spark-3-0_2.12-1.0.0.jar dotnet bin\Debug\netcoreapp3.1\mySparkApp.dll
Terminal
spark-submit --class org.apache.spark.deploy.dotnet.DotnetRunner --master local bin/Debug/netcoreapp3.1/microsoft-spark-3-0_2.12-1.0.0.jar dotnet bin/Debug/netcoreapp3.1/mySparkApp.dll

If your app runs successfully, you should see the data written to the console:

Command prompt
... logging ...

+------+-----+
|  word|count|
+------+-----+
|  .NET|    3|
|Apache|    2|
|   app|    2|
|  This|    2|
| Spark|    2|
| World|    1|
|counts|    1|
|   for|    1|
| words|    1|
|  with|    1|
| Hello|    1|
|  uses|    1|
+------+-----+

... logging ...

You may encounter an error that Spark failed to delete a jar file from a temporary directory. This is a known error and won't affect the output of your app.

Continue