https://gimletlabs.ai/blog A blog about research on high performance AI systems. en-US hello@gimletlabs.ai (Gimlet Labs Inc.) hello@gimletlabs.ai (Gimlet Labs Inc.) Mon, 20 Oct 2025 00:00:00 GMT https://gimletlabs.ai/blog/heterogeneous-ai-infrastructure https://gimletlabs.ai/blog/heterogeneous-ai-infrastructure AI workloads are shifting from simple LLM inference to complex, multi-model workflows. To run them efficiently at scale, we need a system that can dynamically decompose workloads, plan and schedule them, and map execution to the right hardware. Mon, 20 Oct 2025 00:00:00 GMT hello@gimletlabs.ai (Gimlet Labs Inc.) InferencePerformanceEfficiency https://gimletlabs.ai/blog/multivendor-prefill-decode-disaggregation https://gimletlabs.ai/blog/multivendor-prefill-decode-disaggregation Separating prefill and decode stages of LLM inference improves token throughput because their resource needs differ. Although most deployments use NVIDIA hardware for both stages, multivendor disaggregation can actually improve efficiency while maintaining SLAs. Based on our models using NVIDIA B200s and Intel Gaudi 3, common workloads can see 1.7X TCO improvement compared to single-vendor disaggregation. Mon, 13 Oct 2025 00:00:00 GMT hello@gimletlabs.ai (Gimlet Labs Inc.) InferencePerformanceEfficiencyHardware