Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
ai_slop_hater
16 days ago
|
parent
|
context
|
favorite
| on:
Anonymous request-token comparisons from Opus 4.6 ...
Does anyone know what changed in the tokenizer? Does it output multiple tokens for things that were previously one token?
quux
16 days ago
[–]
It must, if it now outputs more tokens than 4.6's tokenizer for the same input. I think the announcement and model cards provide a little more detail as to what exactly is different
Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
