The sample efficiency of the RL algorithm, even for simple games, is not very good. This usually means that we will need a lot of episodes for the policy to learn to excel. Being able to run policy in an environment that can parallel and accelerate could be very helpful for the improvement - for example running a batch of browsers or tabs simultaneously :)
People born in different eras often develop different worldviews by the time they reach their 50s. Not everyone is lucky enough to be born in the "right time".
I am curious about the setup of 14 GPUs - what kind of platform (motherboard) do you use to support so many PCIe lanes? And do you even have a chassis? Is it rack-mounted? Thanks!
I used a large supermicro server chassis, a dual Xeon motherboard with 7 8 lane PCI Express slots, all the ram it would take (bought second hand), splitters, four massive powersupplies. I extended the server chassis with aluminum angle riveted onto the base. It could be rack mounted but I'd hate to be the person lifting it in. The 3090s were a mix, 10 of the same type (small, and with blower style fans on them) and 4 much larger ones that were kind of hard to accommodate (much wider and longer). I've linked to the splitter board manufacturer in another comment in this thread. That's the 'hard to get' component but once you have those and good cables to go with them the remaining setup problems are mostly power and heat management.
1. Start by learning a simulation tool, e.g. Mujoco (open source) or Isaac Sim.
2. Learn basics of optimal control and reinforcement learning, reproduce papers/ideas in the simulation.
3. Get your hands dirty on a cheap robot, and try deploy your trained model on it. For mobility and manipulation. Unitree Go1/Go2 for mobility, and robotic arms for manipulation.
I think having to debug to find problem of your system is frustrating. But with NixOS, I at least won't be afraid of "breaking the system" or doing something "irreversible". This is totally a peace of mind when tinkering with my setup.
reply