Government AI: Starting Small with Small Language Models (SLMs)

Two Elder Research staff members sit smiling at a desk in front of a laptop

Many government agencies are in the early stages of exploring generative AI and large language models (LLMs). As organizations navigate this discovery phase, our experience suggests a valuable insight: small language models (SLMs) can offer a practical and cost-effective starting point for government organizations venturing into AI.

In this article we share how SLMs can serve as an ideal proof-of-concept to help government agencies assess potential AI solutions more efficiently and cost-effectively.

What Are Small Language Models?

Large AI models like GPT-4 are impressive but require enormous computing power. Small language models are more compact and efficient, designed for specific tasks or domains.

Currently, SLMs typically have fewer than 10 billion parameters—the individual pieces of information the model learns from during training. This makes them more manageable and easier to deploy than their larger counterparts.

Closeup side view image of hands working on laptop

LLMs vs. SLMs: Key Differences

While the threshold between large and small continues evolving as models are rapidly being released, there are key distinctions.

Size and Capabilities

LLMs are massive and intended to handle a wide range of tasks across multiple domains (e.g., GPT-3, GPT-4).

SLMs are smaller and often designed for specific tasks or domains (e.g., Mistral 7B, Microsoft’s Phi-2, Google’s Gemma).

Resource Requirements and Deployment

LLMs need substantial computational resources and are often accessed via cloud APIs.

SLMs can run on more modest hardware and be deployed on-premises, offering greater control over data and security.

Training and Fine-tuning

LLMs are resource-intensive to train and fine-tune.

SLMs are easier to adapt to specific use cases or domains. Through careful data curation, SLMs can even rival the performance of larger language models.

Cost

LLMs have substantial computational costs for running and maintenance.

SLMs are generally more cost-effective, making them suitable for budget-conscious agencies.

AI Use Case: Leveraging SLMs for EDGAR Filings

Two Elder Research staff members walk down a hall at the office in Arlington, VA.

A common challenge for government agencies is effectively using unstructured data for analysis. Our government client faces this challenge with EDGAR filings—an online corporate filing system created by the Securities and Exchange Commission to enhance transparency and accessibility of corporate information.

One of our government clients aimed to explore AI as a potential solution to help make the vast amount of unstructured EDGAR data more manageable and useful for their analysts. However, two main challenges emerged.

The first challenge was the lack of precedent from enterprise IT or governance policies on AI projects. The second issue was the potentially high costs associated with hosting large language models due to their significant computational requirements.

In consideration of these challenges, our client decided to start small by implementing Mistral 7B, a small language model. This strategic choice allowed for in-house deployment on an on-premises server, ensuring government control of access and requiring lower computational resources than LLMs.

Our team leveraged the Mistral 7B model in several steps, accessing it through REST API calls:

Preprocessing: We extracted and standardized relevant subsections within documents, ensuring high-quality input data for the model.

Prompt Engineering: We iteratively refined our queries to improve the consistency and success rate of the model’s outputs.

Output Validation: We employed LangChain’s Pydantic Parser to enforce structured and consistent model responses.

While there’s room for improvement in achieving the desired output structure, this trial demonstrated the SLM’s efficacy in straightforward text parsing tasks for EDGAR filings.

Through the insights from this project, the client gained a solid foundation for scaling to more complex AI solutions.

Elder Research team member sits at a desk working on a computer

Potential Use Cases for SLMs in Government Agencies

Other use cases can benefit from SLMs’ ability to perform specialized, well-defined tasks within a specific domain. SLMs can be efficiently trained or fine-tuned for these focused applications, offering faster processing times, easier deployment, and greater security control with on-premises deployment, compared to larger models.

Information Extraction and Classification

Named Entity Recognition in specific domains (e.g., Census Bureau processing survey responses)

Document classification for specialized departments (e.g., Patent and Trademark Office sorting applications)

Form field extraction (e.g., IRS processing tax forms)

Analysis and Compliance

Targeted sentiment analysis (e.g., Department of Veterans Affairs gauging veteran satisfaction with services)

Compliance checking for specific regulations (e.g., Environmental Protection Agency screening environmental impact reports)

Information Management and Access

Automated metadata generation (e.g., National Archives and Records Administration enhancing document cataloging)

Text summarization for domain-specific reports (e.g., National Institutes of Health summarizing research grant proposals)

User Interaction

Specialized chatbots for department-specific queries (e.g., Citizenship and Immigration Services addressing visa-related questions)

Conclusion

As government agencies continue exploring the potential of AI, SLMs offer a pragmatic approach to harnessing this technology. They allow organizations to start small, learn fast, and scale intelligently.

By beginning with SLMs, government entities can develop the expertise and infrastructure necessary for more ambitious AI initiatives, ultimately leading to more efficient and effective public services.

Government AI: Starting Small with Small Language Models (SLMs)

Author:

Date Published:

What Are Small Language Models?

LLMs vs. SLMs: Key Differences

Size and Capabilities

Resource Requirements and Deployment

Training and Fine-tuning

Cost

AI Use Case: Leveraging SLMs for EDGAR Filings

Potential Use Cases for SLMs in Government Agencies

Information Extraction and Classification

Analysis and Compliance

Information Management and Access

User Interaction

Conclusion

Government AI: Starting Small with Small Language Models (SLMs)

Author:

Date Published:

What Are Small Language Models?

LLMs vs. SLMs: Key Differences

Size and Capabilities

Resource Requirements and Deployment

Training and Fine-tuning

Cost

AI Use Case: Leveraging SLMs for EDGAR Filings

Potential Use Cases for SLMs in Government Agencies

Information Extraction and Classification

Analysis and Compliance

Information Management and Access

User Interaction

Conclusion

Related Topics

The Modern-Day AI Executive: Defining and Demystifying AI

The Modern-Day AI Executive: Most AI Investments Return Zero

Reflections on Generative AI

Author

Harrison Blondeau

Data Scientist

Author

John Moroney

Data Scientist