Most LLM code you find online works great in a Jupyter notebook and falls apart the moment real traffic hits it. The bugs aren't exotic — they're small, structural decisions that don't show up until something goes wrong at the worst possible time. Here are six of them, and the fix for each. Don't hide streaming behind a boolean flag It's tempting to write one function with an if stream: branch. Don't. It means every caller has to know, out of band, which type they're going to get back — a plain string or a generator — and a wrong guess fails silently or crashes deep in your UI code. Split it into two functions instead: python def ask(prompt: str) -> str: """Returns the full text response.""" ... def ask_streaming(prompt: str): """Yields text chunks as they arrive.""" ... Now the function signature is the documentation. ask() gives you something easy to log, cache, and unit test. ask_streaming() is unambiguously a generator, built for UIs. No flag, no ambiguity. Wrap streaming calls in a context manager If your loop over a streaming response throws an exception halfway through — network hiccup, a bug in your own rendering code, whatever — does the underlying HTTP connection actually close? With a naive implementation, often not. Connections leak quietly until you're wondering why your process is hoarding sockets. The fix is boring but effective: python with client.messages.stream(...) as stream: for text in stream.text_stream: yield text The context manager guarantees cleanup runs even on an error mid-stream. This is a one-line difference that only matters the day something actually goes wrong — which is exactly when you don't want to be debugging a connection leak too. Trim chat history on turn pairs, not message count A common way to cap conversation memory is to just keep "the last N messages." The bug: if N happens to land in the middle of a user/assistant pair, you end up with an orphaned assistant message with no matching user turn — and some APIs will outright reject that payload. Slice on pairs instead: python def _trimmed_messages(self) -> list[dict]: recent = self.history[-(self.max_history_turns * 2):] return [{"role": "system", "content": self.system_prompt}] + recent Multiplying by 2 and slicing keeps whole user→assistant exchanges intact, so you never truncate mid-conversation in a way the API can't parse. Normalize your vectors before cosine similarity search This one is sneaky because it fails silently. If you're using FAISS's IndexFlatIP (inner product) to approximate cosine similarity, that approximation is only valid if your vectors are unit-normalized first. Skip it, and you don't get an error — you get search results ranked in the wrong order, with no signal that anything's broken. python @staticmethod def _normalize(vectors: np.ndarray) -> np.ndarray: norms = np.linalg.norm(vectors, axis=1, keepdims=True) return vectors / np.clip(norms, 1e-10, None) If your semantic search results look "close but weirdly off," this is one of the first things worth checking. Only retry the errors that are actually transient Blanket "retry on any exception" logic is a trap. If a request fails because you sent a malformed parameter, retrying it four times just guarantees the same failure four times — burning latency and rate-limit budget for nothing. Scope retries to the errors that can plausibly resolve themselves: python @retry( stop=stop_after_attempt(4), wait=wait_exponential(multiplier=1, min=1, max=20), retry=retry_if_exception_type((RateLimitError, APITimeoutError, APIError)), reraise=True, ) def _call_model(...): ... Rate limits and timeouts are worth retrying with backoff. Your own bugs are not — those should fail fast so you actually see them. Give yourself a fallback model, not just a retry loop Retries handle transient failures. They don't help if a model is degraded or down for an extended window. Pairing retries with a fallback to a secondary model is the same pattern most production LLM gateways use under the hood: python try: return _call_model(primary_model, messages) except RETRYABLE_ERRORS: return _call_model(fallback_model, messages) Cheap/fast model first, stronger model as a safety net. Your app degrades gracefully instead of just erroring out. Where these came from I pulled these six lessons out of The AI & LLM Integration Cookbook — a set of 10 complete, copy-pasteable Python templates covering both the OpenAI and Anthropic APIs (quick-starts, tool calling, RAG over PDFs with LangChain and LlamaIndex, embeddings/vector search, and the production wrapper above). Each recipe includes a "why this pattern" note like the ones above, so you're not just getting code — you're getting the reasoning that usually only shows up after something's already broken in production once. Worth a look if you're wiring up your first LLM feature or auditing an existing integration for exactly these kinds of gaps. Get the AI and LLM cookbook at 20% off product link- https://payhip.com/besttechbooks