ReinforcementLearning.jl,as the name says, is a package for reinforcement learning research in Julia.
Our design principles are:
- Reusability and extensibility: Provide elaborately designed components andinterfaces to help users implement new algorithms.
- Easy experimentation: Make it easy for new users to run benchmarkexperiments, compare different algorithms, evaluate and diagnose agents.
- Reproducibility: Facilitate reproducibility from traditional tabularmethods to modern deep reinforcement learning algorithms.
��
Get Started
julia> ] add ReinforcementLearning
julia> using ReinforcementLearning
julia> run(
RandomPolicy(),
CartPoleEnv(),
StopAfterStep(1_000),
TotalRewardPerEpisode()
)
The above simple example demonstrates four core components in a generalreinforcement learning experiment:
Policy. The
RandomPolicyis the simplest instance ofAbstractPolicy.It generates a random action at each step.Environment. The
CartPoleEnvis a typicalAbstractEnvto test reinforcement learning algorithms.Stop Condition. The
StopAfterStep(1_000)is to inform that our experiment should stop after1_000steps.Hook. The
TotalRewardPerEpisodestructure is one of the most commonAbstractHooks.It is used to collect the total reward of each episode in an experiment.
Check out the tutorial page to learn how these fourcomponents are assembled together to solve many interesting problems. We alsowrite blog occasionally toexplain the implementation details of some algorithms. Among them, the mostrecommended one is An Introduction toReinforcementLearning.jl,which explains the design idea of this package. Besides, a collection ofexperiments are also provided to help you understand how to trainor evaluate policies, tune parameters, log intermediate data, load or saveparameters, plot results and record videos. For example:
��
Project Structure
ReinforcementLearning.jl itself is just a wrapper around several othersubpackages. The relationship between them is depicted below:
+-----------------------------------------------------------------------------------+ | | | ReinforcementLearning.jl | | | | +------------------------------+ | | | ReinforcementLearningBase.jl | | | +----|-------------------------+ | | | | | | +--------------------------------------+ | | +---->+ ReinforcementLearningEnvironments.jl | | | | +--------------------------------------+ | | | | | | +------------------------------+ | | +---->+ ReinforcementLearningCore.jl | | | +----|-------------------------+ | | | | | | +-----------------------------+ | | +---->+ ReinforcementLearningZoo.jl | | | +----|------------------------+ | | | | | | +-------------------------------------+ | | +---->+ DistributedReinforcementLearning.jl | | | +-------------------------------------+ | | | +------|----------------------------------------------------------------------------+ | | +-------------------------------------+ +---->+ ReinforcementLearningExperiments.jl | | +-------------------------------------+ | | +----------------------------------------+ +---->+ ReinforcementLearningAnIntroduction.jl | +----------------------------------------+
��
Supporting
ReinforcementLearning.jl is a MIT licensed open source project with itsongoing development made possible by many contributors in their spare time.However, modern reinforcement learning research requires huge computingresource, which is unaffordable for individual contributors. So if you or yourorganization could provide the computing resource in some degree and would liketo cooperate in some way, please contact us!
✍️
Citing
If you use ReinforcementLearning.jl in a scientific publication, we wouldappreciate references to the CITATION.bib.
✨
Contributors
Thanks goes to these wonderful people (emoji key):
This project follows the all-contributors specification. Contributions of any kind welcome!