An app that can predict whether the text from customer reviews is negative or positive sentiment.
Download and install
Model Builder ships with Visual Studio version 16.6.1 and later versions when you install one of the .NET workloads.
Open the Visual Studio Installer and Modify the instance of Visual Studio that you'd like to use with Model Builder.
Check the optional ML.NET Model Builder (Preview) component in any of the .NET workloads (such as .NET Core cross-platform development).
Enable Preview Feature
ML.NET Model Builder is currently a Preview feature. So, you must enable this Preview feature in Visual Studio in order to use the tool.
In Visual Studio, go to Tools > Options > Environment > Preview Features.
Check Enable ML.NET Model Builder.
Note: If the tutorial screenshots do not match with what you see, you may need to update your version of Model Builder. Go to Extensions > Manage Extensions to make sure that there are no available updates for Model Builder. The most recent version is 16.3.0.2056001.
Install .NET SDK
To build .NET apps, you need to download and install the .NET Core 3.1 SDK (Software Development Kit).
The ML.NET command-line interface (CLI), provides tools for building machine learning models with ML.NET.
Note: ML.NET CLI is currently in Preview.
Once you've installed the .NET SDK, open a new terminal and run the following command:
Command prompt
dotnet tool install -g mlnet
If the tool installs successfully, you should see the following output message:
Command prompt
You can invoke the tool using the following command: mlnetTool 'mlnet' (version '16.1.1') was successfully installed.
Note: If you're using a console other than Bash (for example, zsh, which is the new default for macOS), then you'll need to give mlnet executable permissions and include mlnet to the system path. Instructions on how to do this should appear in the terminal when you install mlnet (or any global tool).
Alternatively, you can try using the following command to run the mlnet tool:
Command prompt
~/.dotnet/tools/mlnet
If the command still gives you an error, use the I ran into an issue button below to report the issue and get help fixing the problem.
Create your app
Open Visual Studio and create a new .NET Core console app:
Select Create a new project from the Visual Studio 2019 start window.
Choose the C# Console App (.NET Core) project template.
Change the project name to myMLApp.
Make sure Place solution and project in the same directory is unchecked.
Create the project.
Add machine learning
Right-click on the myMLApp project in Solution Explorer and select Add > Machine Learning.
This opens ML.NET Model Builder in a new docked tool window in Visual Studio. Model Builder will guide you through the process of building a machine learning model in the following steps.
In your terminal, run the following commands:
Command prompt
mkdir myMLAppcd myMLAppdotnet new console -o consumeModelApp
The mkdir command creates a new directory named myMLApp, and the cd myMLApp command puts you into the newly created app directory. The dotnet command creates a new C# application of type console for you. The -o parameter creates a directory named consumeModelApp where your app is stored, and populates it with the required files.
Your model training code will be generated in the upcoming steps.
Pick a scenario
To generate your model, you first need to select your machine learning scenario. Model Builder supports several scenarios:
In this case, you'll predict sentiment based on the content (text) of customer reviews.
In the Model Builder Scenario screen, select the Text classification scenario, since you're predicting which category a comment falls into (positive or negative).
After selecting the Text classification scenario, you must choose your training environment. While some scenarios support training in Azure, Classification currently only supports local training, so keep the Local environment selected and move on to the Data step.
To generate your model, you need to select your machine learning scenario.
There are several ML scenarios that are supported by the ML.NET CLI:
Classification - Use this when you want to predict which category data belongs in (for example, analyzing sentiment of customer reviews as either positive or negative).
Image classification - Use this when you want to predict which category an image belongs to (for example, predicting if an image is of a cat or a dog).
Regression (for example, value prediction) - Use this when you want to predict a numeric value (for example, predicting house price).
Recommendation - Use this when you want to recommend items to users based on historical ratings (for example, product recommendation).
In this case, you'll predict sentiment based on the content (text) of customer reviews, so you'll use classification.
Each row in yelp_labelled.txt represents a different review of a restaurant left by a user on Yelp. The first column represents the comment left by the user, and the second column represents the sentiment of the text (0 is negative, 1 is positive). The columns are separated by tabs, and the dataset has no header. The data looks like the following:
yelp_labelled.txt
Wow... Loved this place. 1Crust is not good. 0Not tasty and the texture was just nasty. 0
Add data
In Model Builder, you can add data from a local file or connect to a SQL Server database. In this case, you'll add yelp_labelled.txt from a file.
Select File as the input data source type.
Browse for yelp_labelled.txt. Once you select your dataset, a preview of your data appears in the Data Preview section. Since your dataset does not have a header, headers are auto-generated ("col0" and "col1").
Under Column to predict (Label), select "col1." The Label is what you're predicting, which in this case is the sentiment found in the second column ("col1") of the dataset.
The columns that are used to help predict the Label are called Features. All of the columns in the dataset besides the Label are automatically selected as Features. In this case, the review comment column ("col0") is the Feature column. You can update the Feature columns and modify other data loading options in Advanced data options, but it is not necessary for this example.
After adding your data, go to the Train step.
Train your model
Now you'll train your model with the yelp_labelled.txt dataset.
Model Builder evaluates many models with varying algorithms and settings based on the amount of training time given to build the best performing model.
Change the Time to train, which is the amount of time you'd like Model Builder to explore various models, to 20 seconds (you can try increasing this number if no models are found after training) . Note that for larger datasets, the training time will be longer. Model Builder automatically adjusts the training time based on the dataset size.
Select Start training to start the training process. Once training starts, you can see the time remaining.
Training results
Once training is done, you can see a summary of the training results.
Best accuracy - This shows you the accuracy of the best model that Model Builder found. Higher accuracy means the model predicted more correctly on test data.
Best model - This shows you which algorithm performed the best during Model Builder's exploration.
Training time - This shows you the total amount of time that was spent training / exploring models.
Models explored (total) - This shows you the total number of models explored by Model Builder in the given amount of time.
If you want, you can view more information about the training session in the Machine Learning Output window.
After model training finishes, go to the Evaluate step.
In your terminal, run the following command (in your myMLApp folder):
The mlnet classification command runs ML.NET with AutoML to explore many iterations of classification models in the given amount of train time with varying combinations of data transformations, algorithms, and algorithm options and then chooses the highest performing model.
--dataset: You chose yelp_labelled.txt as the dataset (internally, the CLI will split the one dataset into training and testing datasets).
--has-header: You use this option to specify if the dataset has a header. In this case, the dataset does not have a header, so it is false
--label-col: You must specify the target column you want to predict (or the Label). In this case, you want to predict the sentiment in the second column (zero-indexed columns means this is column "1").
--train-time: You must also specify the amount of time you would like the ML.NET CLI to explore different models, in this case 20 seconds (you can try increasing this number if no models are found after training). Note that for larger datasets, you should set a longer training time.
Progress
While the ML.NET CLI is exploring different models, it displays the following data:
Start training - This section shows each model iteration, including the trainer (algorithm) used and evaluation metrics for that iteration.
Time left - This and the progress bar will indicate how much time is left in the training process in seconds.
Best algorithm - This shows you which algorithm has performed the best so far.
Best score - This shows you the performance of the best model so far. Higher accuracy means the model predicted more correctly on test data.
If you want, you can view more information about the training session in the log file generated by the CLI.
Evaluate your model
The Evaluate step shows you the best-performing algorithm and the best accuracy and lets you try out the model in the UI.
Try out your model
You can make predictions on sample input in the Try your model section. The textbox is pre-filled with the first line of data from your dataset, but you can change the input and hit Predict to try out different sentiment predictions.
In this case, 0 means negative sentiment and 1 means positive sentiment.
After evaluating and trying out your model, move on to the Consume step.
After the ML.NET CLI selects the best model, it will display the training Summary, which shows you a summary of the exploration process, including how many models were explored in the given training time.
Top models
While the ML.NET CLI generates code for the highest performing model, it also displays the top models (up to 5) with the highest accuracy that it found in the given exploration time. It displays several evaluation metrics for those top models, including AUC, AUPRC, and F1-score. For more information, see ML.NET metrics.
Generate code
In the Consume step in Model Builder, select Add to solution.
Model Builder adds both the machine learning model and the projects for training and consuming the model to your solution. In the Solution Explorer, you should see the code files that were generated by Model Builder.
myMLAppML.ConsoleApp is a .NET Core console app that contains ModelBuilder.cs (used to build/train the model) and Program.cs (used to test run the model).
myMLAppML.Model is a .NET Standard class library that contains ModelInput.cs and ModelOutput.cs (input/output classes for model training and consumption), ConsumeModel.cs (class that contains the method for model consumption), and MLModel.zip (the trained serialized ML model).
The ML.NET CLI adds both the machine learning model and the projects for training and consuming the model to your solution, including:
A .NET console app (SampleClassification.ConsoleApp), which contains ModelBuilder.cs (used to build/train the model) and Program.cs (used to run the model).
A .NET Standard class library (SampleClassification.Model), which contains ModelInput.cs and ModelOutput.cs (input/output classes for model training and consumption) and MLModel.zip (generated serialized ML model).
To try the model, you can run the console app (SampleClassification.ConsoleApp) to predict the sentiment of a single statement with the model.
Consume your model
Model Builder generates the trained model and code for you, including the next steps for model consumption, so you can easily use the model in your end-user .NET application.
You can see an example of how to consume the model in the Program.cs file in the myMLAppML.ConsoleApp project.
Replace the Program.cs code in your myMLApp project with the following code:
Program.cs
using System;using MyMLAppML.Model;namespace myMLApp{ class Program { static void Main(string[] args) { // Add input data var input = new ModelInput() { Col0 = "This restaurant was wonderful." }; // Load model and predict output of sample data ModelOutput result = ConsumeModel.Predict(input); // If Prediction is 1, sentiment is "Positive"; otherwise, sentiment is "Negative" string sentiment = result.Prediction == "1" ? "Positive" : "Negative"; Console.WriteLine($"Text: {input.Col0}\nSentiment: {sentiment}"); } }}
Run myMLApp (select Ctrl+F5 or Debug > Start Without Debugging). You should see the following output, predicting whether the input statement is positive or negative.
The ML.NET CLI has generated the trained model and code for you, so you can now use the model in your other .NET applications (for example, your consumeModelApp console app) by following these steps:
Copy the generated MLModel.zip file from the SampleClassification.Model folder to the consumeModelApp folder.
In the command line, navigate to the consumeModelApp directory.
Command prompt
cd consumeModelApp
Add a reference to the generated class library that contains the trained model and consumption code (SampleClassification.Model) in your consumeModelApp project by running the following command:
Open the consumeModelAppProgram.cs in any code editor and replace the code with the following:
Program.cs
using System;using SampleClassification.Model;namespace consumeModelApp{ class Program { static void Main(string[] args) { // Add input data var input = new ModelInput() { Col0 = "This restaurant was wonderful." }; // Load model and predict output of sample data ModelOutput result = ConsumeModel.Predict(input); // If Prediction is 1, sentiment is "Positive"; otherwise, sentiment is "Negative" string sentiment = result.Prediction == "1" ? "Positive" : "Negative"; Console.WriteLine($"Text: {input.Col0}\nSentiment: {sentiment}"); } }}
Run your consumeModelApp. You can do this by running the following command in the terminal (make sure you are in the consumeModelApp directory):
Command prompt
dotnet run
The output should look something like this:
Next steps
Congratulations, you've built your first machine learning model with ML.NET Model Builder!
To keep learning, try out our Price Prediction tutorial on Docs:
Congratulations, you've built your first machine learning model with the ML.NET CLI!
Now that you've used the ML.NET CLI for Classification (specifically sentiment analysis), you can try other scenarios. Try out a Regression scenario (specifically price prediction) using the Taxi Fare dataset to keep building ML.NET models with the ML.NET CLI.
