I was going to suggest implementing RoPE to fix the context limit, but realized ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		drincanngao 20 days ago \| parent \| context \| favorite \| on: Show HN: I built a tiny LLM to demystify how langu... I was going to suggest implementing RoPE to fix the context limit, but realized that would make it anatomically incorrect.

armanified 20 days ago [–]

I intentionally removed all optimizations to keep it vanilla.

Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact