LLM Operations Engineer.

LLM operations deals with unique challenges: much larger model sizes (billions of parameters), specialized hardware requirements (multi-GPU inference), different evaluation methodologies (human preference, automated rubrics), prompt management as a deployment artifact, and cost optimization at a different scale. Traditional MLOps skills transfer, but LLM-specific expertise is additionally required.

Core tools include vLLM, TGI, or TensorRT-LLM for inference serving, along with observability platforms like LangSmith or Weights and Biases. Knowledge of Kubernetes for GPU orchestration, model registries, and A/B testing frameworks for model versions is essential. Cost monitoring and optimization tooling specific to GPU workloads is also important.

In Austin, large enterprises typically pay LLM Operations Engineers 10 to 20% more in base salary than small companies, but startups often compensate with equity that can exceed base salary value. Equity at AI infrastructure companies adds $25,000 to $80,000 annually. The $125,000 to $200,000 range reflects this entire spectrum.

Texas has no state income tax, which can mean 5 to 10% more take home pay compared to California roles. Property taxes are above average, however, running about 1.8% of home value. When comparing offers across states, your take home pay matters more than the number on the offer letter. A lower salary in a no income tax state can net more than a higher one elsewhere.

Entry level LLM Operations Engineer positions in Austin typically start near $125,000. Candidates with relevant internships, certifications, or portfolio work often negotiate closer to the median of $161,000. Junior LLM ops engineers start at $105,000 to $125,000 managing basic model deployments.

Junior LLM ops engineers start at $105,000 to $125,000 managing basic model deployments. Mid-level engineers optimizing inference infrastructure reach $145,000 to $170,000. Senior engineers designing multi-model serving platforms earn $180,000 to $220,000. Leading LLM infrastructure at major AI companies pushes total comp above $280,000. In Austin, each step up the ladder is amplified by the local cost of living multiplier, which means senior roles pay proportionally more than in lower cost markets.

Ask about equity structure, vesting schedule, annual bonus targets, 401(k) match, health insurance premiums, PTO policy, and remote flexibility. Equity at AI infrastructure companies adds $25,000 to $80,000 annually. In Austin's market, these extras can add $40,250 or more on top of the base salary.