We’d love to start by saying that we really appreciate your interest in Caffe2, and hope this will be a high-performance framework for your machine learning product uses. Caffe2 is intended to be modular and facilitate fast prototyping of ideas and experiments in deep learning. Given this modularity, note that once you have a model defined, and you are interested in gaining additional performance and scalability, you are able to use pure C++ to deploy such models without having to use Python in your final product. Also, as the community develops enhanced and high-performance modules you are able to easily swap these modules into your Caffe2 project.
Pick Your Path
- Use a pre-trained neural network off the shelf! (Easy)
- Make my own neural network! (Intermediate)
- Mobile First! I want to make an app that uses deep learning! (Advanced)
If you chose 1, click the link to where several examples are using pre-trained models and we will show you how to get a demo project up and running in minutes.
If you chose 2 then you’ll need some background in neural networking first. Have that dialed in already? Skip ahead to the link. Need a primer or a refresher? Some resources are listed below.
If you chose 3, click the link to discover how to have image classification in your Android or iOS app. It’s pretty much plug-n-play with Android Studio or Xcode, but you’ll need to integrate directly with Caffe2’s C++ hooks.
New to deep learning
A broad introduction is given in the free online draft of Neural Networks and Deep Learning by Michael Nielsen. In particular the chapters on using neural nets and how backpropagation works are helpful if you are new to the subject.
For an exposition of neural networks in circuits and code, check out Hacker’s Guide to Neural Networks by Andrej Karpathy (Stanford).
Experienced researchers in some facet of machine learning
The Tutorial on Deep Learning for Vision from CVPR ‘14 is a good companion tutorial for researchers. Once you have the framework and practice foundations from the Caffe tutorial, explore the fundamental ideas and advanced research directions in the CVPR ‘14 tutorial.
These recent academic tutorials cover deep learning for researchers in machine learning and vision:
- Deep Learning Tutorial by Yann LeCun (NYU, Facebook) and Marc’Aurelio Ranzato (Facebook). ICML 2013 tutorial.
- LISA Deep Learning Tutorial by the LISA Lab directed by Yoshua Bengio (U. Montréal).
IPython Notebook Tutorials and Example Scripts
The IPython notebook tutorials and example scripts we have provided below will guide you through the Caffe2 Python interface. Some tutorials have been generously provided by the Caffe community and we welcome more contributions of this kind to help others get ramped up more quickly and to try out the many different uses of Caffe2. The iPython notebook tutorials can be browsed or downloaded using the links below each tutorial’s title. You may browse these ipynb files on Github directly and this is the preferred route if you just want to look at the code and try it out for yourself. However, it is recommended to run them in Jupyter Notebook and take advantage of their interactivity. Installation instructions below will show you how to do this. Skip this part if you want to jump right into the tutorial descriptions below.
There are example scripts that can be found in /caffe2/python/examples that are also great resources for starting off on a project using Caffe2.
- char_rnn.py: generate a recurrent convolution neural network that will sample text that you input and randomly generate text of a similar style
- lmdb_create_example.py: create an lmdb database of random image data and labels that can be used a skeleton to write your own data import
- resnet50_trainer.py: parallelized multi-GPU distributed trainer for Resnet 50. Can be used to train on imagenet data, for example
- seq2seq.py: create a specialized RNN that handles lines of text for projects such as language translation
- seq2seq_util.py: utility functions for the sequence to sequence example script
New to Caffe and Deep Learning? Start here and find out more about the different models and datasets available to you.
Take advantage of the Model Zoo and grab some pre-trained models and take them for a test drive. This tutorial has a set of different models that are ready to go and will show you the basic steps for prepping them and firing up your neural net. Then you can throw some images or other tests at them and see how they perform.
New to Caffe2
You also may want to review the Intro Tutorial before starting this notebook.
This tutorial introduces a few basic Caffe2 components:
This tutorial shows how to use more Caffe2 features with simple linear regression as the theme.
- generate some sample random data as the input for the model
- create a network with this data
- automatically train the model
- review stochastic gradient descent results and changes to your ground truth parameters as the network learned
Learn how to get your images ready for ingestion into pre-trained models or as test images against other datasets. From cell phones to web cams to new medical imagery you will want to consider your image ingestion pipeline and what conversions are necessary for both speed and accuracy during any kind of image classification.
- HWC to CHW
- RGB to BRG
- image prep for Caffe2 ingestion
Creating a Convolutional Neural Network from Scratch
This tutorial creates a small convolutional neural network (CNN) that can identify handwriting. The train and test the CNN, we use handwriting imagery from the MNIST dataset. This is a collection of 60,000 images of 500 different people’s handwriting that is used for training your CNN. Another set of 10,000 test images (different from the training images) is used to test the accuracy of the resulting CNN.
Try your hand at importing and massaging data so it can be used in Caffe2. This tutorial uses the Iris dataset.
Write Your Own Tutorial!
Have a great tutorial that you’ve created or have some ideas? Let’s chat about it. Create an Issue and post a link to your tutorial or post your idea.
Tour of Caffe Components
- gpu.h: needs documentation
- db.h: needs documentation
- TensorProtosDBInput: needs documentation
One of basic units of computation in Caffe2 are the Operators.
Writing Your Own Operators
Fantastic idea! Write custom operators and share them with the community! Refer to the guide on writing operators:
MacOSx via Brew & pip
1 2 3
brew install matplotlib pip install ipython notebook pip install scikit-image
Anaconda comes with iPython notebook, so you’ll only need to install matplotlib.
conda install matplotlib conda install scikit-image
1 2 3
pip install matplotlib pip install ipython notebook pip install scikit-image
Hit it with a hammer you say? Ok, here’s a full list for your installation pleasure, and note that you’ll want them installed with Python, and in some cases C++ or at the system level, so you might
brew install as well as
pip install or
conda install them:
flask graphviz hypothesis jupyter leveldb lmdb matplotlib pydot pyyaml requests scikit-image scipy tornado zeromq
Instructions on how to setup Jupyter Notebook, which is the latest, greatest way to use and create interactive code notebooks (ipynb files) is found at http://jupyter.org.
Note: if you’ve already successfully installed Caffe2 with Anaconda Python, then great news! You already have Jupyter Notebook. Starting it is easy.
Run the shell script included in the
Or you can run
jupyter notebook, and when your browser opens with your local Jupyter server (default is http://localhost:8888), browse to the Caffe2 repository and look for them in the
caffe2/caffe2/python/tutorials directory. Opening them this way will launch their interactive features just like the shell script mentioned above. The script has the additional feature of setting your PYTHONPATH environment variable.