Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM
Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM — from Hacker News front page
toolmodelinfraproduct
AI Infrastructure
AI infrastructure is the backbone of every model deployment — inference optimization, training platforms, GPU allocation, and cloud services change frequently enough that builders and operators need a dedicated signal feed to stay ahead.
This cluster covers infrastructure launches, inference engine updates, training platform features, deployment tool releases, and cloud-service AI signals. Each signal links to the primary source for technical evaluation.
Use this page to monitor infra shifts: catch new inference engines, compare training platform costs, and evaluate deployment tools before committing to a stack.
14 signalsSorted by date, newest first
Jensen Huang’s departure blasts big hole in AI startup funding, VCs and founders say
Nvidia poured tens of billions into AI startups under Huang. Without him at the helm, founders and VCs are bracing for smaller checks and lower valuations.
infraproduct
Export-banned Nvidia chips are behind China’s most powerful AI models — uh-oh, Jensen!
New research shows the most advanced AI models from China were trained on banned Nvidia chips.
modelinfraresearch
The unfolding Jensen Huang scandal: a timeline
Jensen Huang has resigned from Nvidia following an affair, which could have ramifications for national security as the U.S. and China race to develop the most advanced AI infrastr…
infrasecurity
Nebius tests the neocloud playbook with $40 billion in planned spending
Nebius plans to fund its $40 billion AI infrastructure expansion through customer prepayments and loans, even as delays at a Microsoft data center raise questions about its financ…
modelinfra
AgentSSL: Can MLE Agents Leverage Unlabeled Data?
Applying machine learning systems to new domains introduces bottlenecks in both labeled data acquisition and manual pipeline engineering. While recent MLE agents show promise in a…
agentinfra
Sketching Language Models for Context Attribution
Context attribution asks how removing parts of a prompt changes a language model's output. We study leave-one-out (LOO) attribution for a prompt of $n$ tokens and a fixed measurem…
toolmodelinfraresearch
Synthesis Smuggling: Maliciously Hiding Inferred Personal Attributes in LLM Outputs
We propose Synthesis Smuggling, a novel threat model in which an adversarial LLM infers sensitive information about a user from routine assistance-seeking interactions and stegano…
toolmodelinfrasecurity
Jensen Huang’s exit puts Nvidia’s support for open source in question
Jensen Huang’s resignation raises questions about Nvidia’s commitment to open source AI, including its Hugging Face acquisition, open-weight models, and advocacy against restricti…
modelinfraproduct
Ballad: Bandit-Based LLM Routing for Automated Heuristic Discovery
Modern automated heuristic discovery (AHD) systems increasingly rival hand-crafted heuristics by leveraging Large Language Models (LLMs) to iteratively mutate and refine solutions…
toolmodelinfra
Scalable Conversational Assessment: Chat-Based Exams with a Two-Agent LLM Pipeline
Oral exams are compelling because (1) students can be asked to provide their reasoning for an answer and (2) their interactive nature permits follow-up questions to resolve ambigu…
toolmodelagentinfraresearch
Introducing Feedback Delays to Non-Delayed Adversarial MDPs: Optimal $\sqrt S$ and $\sqrt K$ Dependencies in Tabular and Linear AMDPs
We study regret minimization in episodic Adversarial Markov Decision Processes (AMDPs) with unknown transitions and bandit feedback. In tabular AMDPs with $S$ states, $A$ actions,…
infrasecurityresearch
Hugging Face CEO vows to keep fighting for open source AI after Jensen Huang’s exit
Hugging Face's CEO pledged to keep fighting for open source AI after Huang's exit, raising questions about Nvidia's $12.9 billion acquisition.
infraproduct
llm-gemini 0.36
Release: llm-gemini 0.36 Gemini 4 Carbon was announced two weeks ago for "a subset of cyber defenders", but today it's available to everyone else . This is Google's best ever Gemi…
toolmodelinfraproduct