We’d like to share the plans for future Caffe2 evolution. Publicly open-sourced over a year ago, Caffe2 is a light-weight and modular framework that comes production-ready with ultimate scaling capabilities for training and deployment. Its mobile capabilities (Caffe2go) support all major generations of hardware and power one of the largest deployments of mobile deep learning with more than 1 billion devices. Over the past year, we worked with many industry partners to add Caffe2 support for their platform and guaranteed the best possible performance regardless of the platform you run on.

At Facebook, where Caffe2 originates, we support both PyTorch and Caffe2 for the wide range of AI use cases. The main focus of Caffe2 development has been performance and cross-platform deployment whereas PyTorch has focused on flexibility for rapid prototyping and research.

In practice, any deep learning framework is a stack of multiple libraries and technologies operating at different abstraction layers (from data reading and visualization to high-performant compute kernels). Over the past year we saw more components of Caffe2 and PyTorch being shared (e.g. gloo, NNPACK, etc). Also, with our investment into interoperability, we built deep integration between frameworks using the shared ONNX model format.

We realized that in order to deliver the best user experience, it makes sense to combine the beneficial traits of Caffe2 and PyTorch into a single package and enable a smooth transition from fast prototyping to fast execution. It’d also improve our developer efficiency by more easily utilizing a shared set of tools.

Caffe2 and PyTorch projects are merging. Over the next few months, we’re planning to deeply integrate components of the frameworks and effectively unite them as a single package. It will combine the flexible user experience of the PyTorch frontend with scaling, deployment and embedding capabilities of the Caffe2 backend. Following is the high-level outline of the plan.

Developers are looking to apply AI to an ever expanding range of use cases. As we look to broaden how people can use AI, we’re thrilled to share our recent collaboration between ARM and Facebook to integrate and optimize Caffe2 for ARM’s Mali Graphics Processing Unit (GPU) hardware.

Reinforcement learning (RL) is an area of machine learning focused on teaching agents a complex relationship between its action and behavior, and maximizing a reward after a duration in an environment. The agent can be a game avatar, recommender system, notification bot, or variety of other systems that make decisions. The reward could be points in a game, or more engagement on a website. Facebook uses RL in different ways, with one example being when to let page owners know how their pages are performing.

Today, we are pleased to announce RL_Caffe2 (https://github.com/caffe2/reinforcement-learning-models), a set of RL libraries built on the Caffe2 platform. Sharing an open-source fork of our Caffe2 RL framework allows us to give back to the community and also collaborate with other institutions as RL finds more applications in industry.

We are excited to share our recent work on supporting a recurrent neural network (RNN).

We did not support RNN models at our open source launch in April. So, over the last several months, we have developed state-of-the-art RNN building blocks to support RNN use cases (machine translation and speech recognition, for example).

Using Caffe2, we significantly improved the efficiency and quality of machine translation systems at Facebook. We got an efficiency boost of 2.5x, which allows us to deploy neural machine translation models into production. As a result, all machine translation models at Facebook have been transitioned from phrase-based systems to neural models for all languages. In addition, several product teams at Facebook, including speech recognition and ads ranking, have started using Caffe2 to train RNN models.

We invite machine learning engineers and researchers to experience Caffe2’s RNN capability. More details about what we implemented and open-sourced for RNN support are outlined below.

After open sourcing Caffe2 at F8 last month, today we are are excited to share our recent work on low precision 16 bit floating point (FP16) training in collaboration with NVIDIA.

Training and deploying AI models is often associated with massive data centers or super computers, with good reason. The ability to continually process, create, and improve models from all kinds of information: images, video, text, and voice, at massive scale, is no small computing feat. Deploying these models on mobile devices so they’re fast and lightweight can be equally daunting. Overcoming these challenges requires a robust, flexible, and portable deep learning framework.

Facebook has been working with others in the open source community to build such a framework. Today, we’re open-sourcing the first production-ready release of Caffe2 - a lightweight and modular deep learning framework emphasizing portability while maintaining scalability and performance.