Combinatorial optimization problems often admit many feasible solutions, yet some are more desirable to users for reasons that are difficult to express as explicit rules or objectives. This topic investigates reinforcement learning from human feedback (RLHF) to learn such preferences from comparisons between solutions. For this we will build on the fully-neural solver AnyCSP which we will train to prefer nice solutions without including this as explicit conditions in the training instance. We will start by letting it learn known rules and then extend toward feedback from real humans if time allows for that. The planned application is shift planning and similar scheduling problems.

Starting literature:

• AnyCSP: One Model, Any CSP (IJCAI 2023), the underlying neural solver
• Deep Reinforcement Learning from Human Preferences (NeurIPS 2017), the foundational approach to learning rewards from pairwise preferences
• PEBBLE (ICML 2021), methods for learning from limited preference feedback