| name | ppo-credit |
| description | Change Ola PPO credit assignment, reward scale, greedy feedback, or log_prob. Use when the user mentions AI策略, PPO, evolution_pref, Hamming, log_prob, reward, or greedy feedback. |
PPO Credit Assignment
Online writes go through learner.step → decode_action → SafetyGuard.filter_action → apply_params_with_rollback. Do not expand ACTION_DIM.
Constraints (do not revert)
encode_params_to_pref is an offset in [-0.2, 0.2] relative to 0.5. fuse_evolution_pref / apply_evolution_pref adds 0.1 * pref. Never mix pref as a [0,1] action (0.8 must not be pulled toward 0.2).apply_params_with_rollback returns (applied, rollback, ...). If applied is false (interval or unchanged params), do not push_transition and do not overwrite last_action.- After
filter_action, re-encode with encode_params_to_action so buffer / last_action match the landed params.
- After a greedy accept, call
learner.sync_last_action and tuning::sync_applied_params so Hamming and LAST_PARAMS follow the verified set.
cpu_usage is [0,1] (global_cpu_usage()/100). Idle gate is < 0.15. Power penalty is > 0.80. GPU busy is 0-100.compute_log_prob_scalar and compute_log_prob_batch both clamp (z-μ)/σ to ±10.- PPO early-stop KL is
E_{a~π_old}[log π_old - log π_new], not the reversed difference.
- Exploration EMA is
0.9*old + 0.1*new (ema_alpha=0.1).
After a stage
Follow session-memory: /home/wlq/wlq/record.sh with WLQ_DIR=/home/wlq/wlq.
