RealFeedApp beta
Training language models
How language models are trained: data, architectures, fine-tuning methods and new research results.
- New theory explains how AI alters research logic
Analysis of interactions with ChatGPT, Gemini, DeepSeek, and Copilot from 2023 to 2026 revealed that AI preserves terminology while altering logical connections and theoretical structures.
- Validator Stack checks genome designs for biological feasibility
The proposed two-part system uses a compiler to translate prompts into technical specs and a validator stack to check sequence integrity and functional plausibility.
- Transformers implement any computable algorithm
Transformer models can implement any computable algorithm because MLPs handle arbitrary computations and attention provides writable memory. The author built a compiler that generates Llama weights to execute programs without training.
- DCOCO evaluates multiple candidate tokens per position
The DCOCO mechanism adds access to previous tokens within draft blocks without breaking parallelism. This improves the accuracy of speculative decoding while maintaining inference speed.
- MASS improves score per output token on research benchmarks
The MASS system optimizes multi-agent workflows and distills collective intelligence to serve as a self-supervision signal, improving scores per output token on research benchmarks.
- SMITH unifies tool creation and use for LLM agents
The new SMITH methodology unifies tool generation and invocation training within a single policy, employing three independent reward axes to correct errors in schema, code, and outcome. Testing was conducted on a 4 billion parameter Qwen3 model.
- Output compression reduces realized cost by 1.4–2.4x
The CAVEWOMAN protocol demonstrates that output compression reduces realized cost by 1.4–3x, while input compression creates a strict lose-lose scenario.
- Scientists warn we have reached the point of no return in AI
Scientists warn that AI will reach an intelligence explosion, growing exponentially smarter while viewing humans as ants and competing for essential resources.
- Five constants govern the program rewriting loop
The method formalizes algorithm search as a Markov decision process, enabling credit assignment to specific code components. This removes the need for manually tuning five constants that govern program evolution.
- Pre-training involves sudden mode-hopping in computation
The study reveals that during pre-training, LLMs abruptly switch between pattern-matching and generalizable intelligence, a phenomenon termed mode-hopping that defies standard optimization dynamics.
