dummy-link

StochDynamicProgramming

A package for discrete-time optimal stochastic control

First Commit

11/10/2015

Last Touched

26 days ago

Commit Count

482 commits

Readme

StochDynamicProgramming

WARNING: This package is currently in development. Any help or feedback is appreciated.

Latest release: 0.3.0

Documentation Build Status Social
Build Status Gitter
Codecov branch

This is a Julia package for optimizing controlled stochastic dynamic system (in discrete time). It offers three methods of resolution :

  • Stochastic Dual Dynamic Programming (SDDP) algorithm.
  • Extensive formulation.
  • Stochastic Dynamic Programming.

It is built upon JuMP

What problem can we solve with this package ?

  • Stage-wise independent discrete noise
  • Linear dynamics
  • Linear or convex piecewise linear cost

Extension to non-linear formulation are under development. Extension to more complex alea dependance are under developpment.

Why Extensive formulation ?

An extensive formulation approach consists in representing the stochastic problem as a deterministic one with more variable and call a standard deterministic solver. Mainly usable in a linear setting. Computational complexity is exponential in the number of stages.

Why Stochastic Dynamic Programming ?

Dynamic Programming is a standard tool to solve stochastic optimal control problem with independent noise. The method require discretisation of the state space, and is exponential in the dimension of the state space.

Why SDDP?

SDDP is a dynamic programming algorithm relying on cutting planes. The algorithm require convexity of the value function but does not discretize the state space. The complexity is linear in the number of stage, and can accomodate higher dimension state than standard dynamic programming. The algorithm return exact lower bound and estimated upper bound as well as approximate optimal control strategies.

Installation

Installing StochDynamicProgramming is an easy process. Open Julia and enter

julia> Pkg.update()
julia> Pkg.add("StochDynamicProgramming")

Usage

IJulia Notebooks will be provided to explain how this package work. A first example on a two dams valley here.

Documentation

The documentation is built with Sphinx, so ensure that this package is installed:

sudo apt-get install python-sphinx

To build the documentation:

cd doc
make html

License

Released under Mozilla Public License (see LICENSE.md for further details).