Available Mixes
Crosshatch offers several mixes optimized for different use cases and languages. Here's an overview of the currently available mixes:
LMSys Coding Leaderboard Mix
This mix is designed for coding tasks and is based on the top models ranked in the LMSys Chatbot Arena for the Coding category. It automatically adjusts to use the best-performing models, ensuring you always have access to top-tier coding assistance.
Composition
| Model Name | Weight % |
|---|---|
chatgpt-4o-latest | 22.32% |
claude-3-5-sonnet-20240620 | 26.61% |
gpt-4o-2024-05-13 | 30.58% |
gemini-1.5-pro-exp-0827 | 20.49% |
Update Frequency: Based on LMSys leaderboard changes
LMSys Overall Leaderboard Mix
This mix is based on the top models ranked in the LMSys Chatbot Arena for the Overall category, making it suitable for a wide range of tasks.
Composition
| Model Name | Weight % |
|---|---|
chatgpt-4o-latest | 22.51% |
gemini-1.5-pro-exp-0827 | 21.54% |
gemini-1.5-pro-exp-0801 | 23.79% |
gpt-4o-2024-05-13 | 32.15% |
Update Frequency: Based on LMSys leaderboard changes
SynthCode Mix
This synthesis mix uses a mixture-of-agents architecture to provide high-quality coding answers. It employs two "proposer" models and an "aggregation" model to synthesize and refine responses.
Composition
| Model Name | Type |
|---|---|
Claude 3.5 Sonnet | Proposer |
GPT-4 Turbo | Proposer |
GPT-4o | Aggregator |
Performance
18% better performance than the current leader in Bigcodebench Instruct Hard
Scored 31.1% (Pass@1) on 148 problems, compared to 26.4% for GPT-4
Note: This mix may have longer response times due to its complex architecture.
SEAL Spanish Mix
This mix is optimized for Spanish language tasks, based on the top models ranked by Scale.ai's SEAL leaderboard for Spanish.
Composition
| Model Name | Weight % |
|---|---|
gpt-4o | 22.51% |
gemini-1.5-pro | 21.54% |
gpt-4-turbo-preview | 23.79% |
Last updated: est. May 2024
Future Mixes
Crosshatch is continually developing new mixes to address diverse user needs. Some potential future mixes include:
Multilingual Mixes: Optimized for various languages and multilingual tasks.
Role-Playing Mixes: Designed for creative writing and character-based interactions.
Vision Mixes: Combining text and image analysis capabilities.
Speech Mixes: Optimized for speech recognition and generation tasks.
Image Generation Mixes: For creating and manipulating images based on text prompts.
These upcoming mixes aim to expand Crosshatch's capabilities across different domains and modalities. Stay tuned for updates on new mix releases and their specific features and performance metrics.
Last updated