AirLLM 70B inference with single 4GB GPU
AirLLM 70B inference with single 4GB GPU — from Hacker News front page
toolmodelinfra
AI Infrastructure
AI infrastructure is the backbone of every model deployment — inference optimization, training platforms, GPU allocation, and cloud services change frequently enough that builders and operators need a dedicated signal feed to stay ahead.
This cluster covers infrastructure launches, inference engine updates, training platform features, deployment tool releases, and cloud-service AI signals. Each signal links to the primary source for technical evaluation.
Use this page to monitor infra shifts: catch new inference engines, compare training platform costs, and evaluate deployment tools before committing to a stack.
15 signalsSorted by date, newest first
Show HN: Nightcrawler – A local AI pentesting agent running on a smartphone
Article URL: https://github.com/garagehq/nightcrawler/ Comments URL: https://news.ycombinator.com/item?id=49154127 Points: 54 # Comments: 19
agentinfra
A Marc Benioff-backed startup thinks AI can solve the AI deployment problem
June emerged from stealth today with a $20 million pre-seed round to make AI adoption simpler.
infraproduct
deepseek-ai/DeepSeek-V4-Flash-0731
<p><strong><a href="https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731">deepseek-ai/DeepSeek-V4-Flash-0731</a></strong></p> The latest release in DeepSeek's V4 family, "wit…
toolmodelagentinfraproduct
OpenAI reportedly finds evidence that more of its agents ran amok
OpenAI has reportedly found evidence of additional agent misbehavior as it looks into the incident that occurred with Hugging Face.
toolagentinfra
ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction
Enterprise workflows increasingly rely on agents for \emph{schema-guided extraction}: given a document and a user-defined schema, the agent faithfully follows the schema to produc…
modelagentinfraproduct
Sam Altman isn’t the only one who wants to pump the brakes on AI
After years of pushing full speed ahead on AI, OpenAI CEO Sam Altman says maybe it’s time for the AI industry to “pace” itself. The comments came just days after one of OpenAI’s o…
toolmodelinfra
FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models
Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a benchmark for inferring whether two people are already familiar or are meeting a…
toolmodelinfra
TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning
The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input-output examples and apply it to a new grid. Looped visual reas…
modelagentinfra
DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat
Games and simulators make valuable benchmarks by turning decisions into measurable outcomes, but many current suites under-test rules-rich tactical reasoning: the ability to choos…
modelagentinfra
AMTFV: Agentic Mathematical Tool-Flow Verification for LLM Self-Correction
Large language models have demonstrated strong mathematical problem-solving capabilities, yet reliably verifying their candidate answers remains challenging. Existing representati…
toolmodelagentinfraresearch
It’s time to panic about AI safety
When the phrase "OpenAI hacked Hugging Face" has more or less entered mainstream culture, you know we have an AI problem. This week, we learned more about exactly how OpenAI's age…
toolagentinfrasecurity
AI labs want to pump the brakes, but Amazon and SpaceX are still blasting off
After years of pushing full speed ahead on AI, OpenAI CEO Sam Altman says maybe it’s time for the AI industry to “pace” itself. The comments came just days after one of OpenAI’s o…
toolmodelinfra
Anthropic says Claude accidentally hacked real companies too
Anthropic just realized several of its Claude AI models hacked into the systems of three different organizations during testing, acting on their own and without the company notici…
toolmodelinfra
Anthropic says its own AI models breached three companies during security tests
After OpenAI's models broke into Hugging Face, Anthropic checked its own history and found three similar incidents.
toolmodelinfrasecurity