Sparse rewards, progress metrics, and preferences are popular, but they

often neglect many aspects of a task
collapse many axes into one measure
frequently yield ambiguity and disagreement across annotators

We instead propose freeform preference learning

","freeContentEncoded","textContent","published","strategy","unknown","locked","deletedAt","isSyncedFromOwnBlog","original","duplicatesCount","duplicateVersion","originalArticleId","errorsCount","appearance","twitter:repost","color","backgroundColor","shortSummary","summary","bullets","persons","places","words","sentiment","subject","meta","eventId","15372ad7643e219051cf299fa20e7780b0466cd78c1eafdf34402eda765bfde2","nostrRootEvent","event",{"_21":222,"_226":227,"_228":116,"_70":229,"_239":28,"_192":240,"_241":242},"sig","b1799d61a6a496d32f19c5a2e846f16735c2ed5de78b485eb399d6262017ee28c1fbf2080625b011ef64b5ab9f7165ad372b620b8a3c3bd8b69f87392ed3dc47","kind",[230,231,235,237],[219,180],[232,233,42,234],"a","30023:983dd797fe72ad62fc8e3ece7eed1f126550972885c8f835d9af5528dad655a3:019f364e-d1e2-1eb4-72c2-2a741bc6c5a4","mention",[236,42],"client",[238,28,42],"p","pubkey","Sparse rewards, progress metrics, and preferences are popular, but they\n\n\n\nnostr:naddr1qvzqqqr4gupzpxpa67tluu4dvt7gu0kw0mk37yn92ztj3pwglq6ant649rddv4drqqjrqvfevcenvdr994jrzefj95ck2c3595mnycej95exzde5x93xxdnrx4snglttakl ","created_at",1783038835,"authorNPub","markdown","Sparse rewards, progress metrics, and preferences are popular, but they\n- often neglect many aspects of a task\n- collapse many axes into one measure\n- frequently yield ambiguity and disagreement across annotators\n\nWe instead propose freeform preference learning\n","freeMarkdown","kind1EventId","kind30023EventId","9fa269d6d17119ed018bb8a7c3129b465b91c5ae7fa3fdbc6d7b97e8774bf148","kind30023",{"_21":249,"_226":252,"_228":253,"_70":254,"_239":28,"_192":266,"_241":242},"3cbaf76b36c6493cec6b589e2507a3951102e02952e3cff71f7f87d7f521a6fa0a27441fc87f19d871b3ef83bd67e798b28fd97bbcacf2ddc69becd217126188",30023,[255,258,259,262,264,265],[256,257],"r","https://bender.layer3.press/articles/019f364e-d1e2-1eb4-72c2-2a741bc6c5a4",[33,180],[260,261],"published_at","1783038835",[263,175],"d",[236,42],[238,28,42],"Sparse rewards, progress metrics, and preferences are popular, but they\n- often neglect many aspects of a task\n- collapse many axes into one measure\n- frequently yield ambiguity and disagreement across annotators\n\nWe instead propose freeform preference learning\n\nhttps://bender.layer3.press/articles/019f364e-d1e2-1eb4-72c2-2a741bc6c5a4","publishedToNostr","htmlContentKind30023","pubDate","2026-07-03T00:33:55.000Z","modifiedAt","2026-07-06T07:22:45.097Z","storyId","tweets",[],"localization","likes","zaps","reposts","comments","relatedArticles",[],"actionData","errors"]