Many machine learning problems can be expressed as stochastic optimization problems: distributed optimization is still in its infancy, and more research is required to improve the performance, the scalability, and the generality of the optimization algorithms. The aim of this project is to study algorithms and system implementations to distribute optimization algorithms across a cluster of cooperating machines.