AWS Launches Nova Forge SDK for Streamlined LLM Customization

What the Nova Forge SDK Does

Amazon Web Services (AWS) has unveiled the Nova Forge SDK, a powerful new toolkit designed to demystify and streamline the customization of large language models (LLMs) for enterprise customers. This SDK addresses a common challenge: while out-of-the-box LLMs are versatile, they often fall short for specific, proprietary, or domain-intensive business needs. The Nova Forge SDK aims to make specialized LLM creation accessible by removing complexities such as dependency management, image selection, and recipe configuration.

The SDK offers a comprehensive suite of customization options, supporting everything from Supervised Fine-Tuning (SFT) and Reinforcement Fine Tuning (RFT) available through Amazon Bedrock, to advanced capabilities in Amazon SageMaker AI, which also includes Direct Preference Optimization (DPO), and both LoRA and full rank-based customization. It acts as a unified toolkit for Nova customers and developers, guiding them through the entire customization lifecycle—from preparing data to managing training jobs and finally deploying their tailor-made models.

At its core, the Nova Forge SDK operates through three distinct layers: an Input Layer where you specify hardware, platform, IAM roles, training methods, data, and hyperparameters; a Customizer Layer that intelligently configures and launches your jobs; and an Output Layer that delivers essential artifacts like Amazon CloudWatch Logs, ML Flow metrics, and the final trained model. These trained models can then be used for further iterative fine-tuning or deployed for inference on Amazon SageMaker AI or Amazon Bedrock. You can learn more about its capabilities by reading the Introducing Nova Forge SDK blog post.

Why It Matters for Enterprises

For businesses, the ability to customize LLMs is no longer a luxury but a necessity. Generic LLMs struggle with proprietary workflows, unique business terminology, and specialized domain knowledge. The Nova Forge SDK directly tackles this by enabling enterprises to build models that deeply understand their specific context, without the heavy lifting traditionally associated with LLM fine-tuning. This dramatically lowers the barrier to entry for developing highly specialized AI applications.

By abstracting away the underlying infrastructure complexities, the SDK allows developers to focus on experimentation and innovation rather than getting bogged down in setup and configuration. It complements existing AWS tools by offering streamlined workflows with smart defaults, while still providing advanced users the flexibility to access the full power of underlying service SDKs when needed. This ensures both ease of use for common tasks and robust control for complex scenarios.

The SDK primarily uses Amazon SageMaker Training Jobs (SMTJ) as its compute platform, leveraging the robust infrastructure of AWS. Training data is typically retrieved from an Amazon Simple Storage Service (Amazon S3) location, ensuring secure and scalable data access for your customization projects.

How to Get Started

Before diving into customizing your LLMs with Nova Forge SDK, there are a few prerequisites. You'll need an active AWS account and the AWS Command Line Interface (CLI) configured. Essential to the process are properly set up IAM roles: a "User role" for running the SDK and AWS CLI, requiring permissions for Amazon SageMaker AI (e.g., CreateTrainingJob), Amazon S3 (read/write to data buckets), Amazon CloudWatch Logs (read), and IAM (PassRole); and an "Execution role" that Amazon SageMaker AI assumes to run training jobs on your behalf. This role's trust policy must allow sagemaker.amazonaws.com to assume it.

When it comes to compute resources, the SDK uses Amazon SageMaker Training Jobs. For demanding tasks, be aware of service quotas. For instance, Nova Lite 2.0 requires a minimum of 4 instances for SFT training, with additional instances potentially needed if running training and evaluation concurrently. Instances like ml.p5.48xlarge are commonly used, so ensure you request sufficient quota for your chosen instance types.

The Nova Forge SDK is designed to simplify what was once a complex process, empowering businesses to create AI models that truly fit their unique needs. For a deeper dive into implementation details and a step-by-step guide, check out the official announcement.

Read more: Introducing Nova Forge SDK to start building your custom enterprise LLMs today.