Mistral v3 Released! Did it Pass the Coding Test?
π Welcome back, AI enthusiasts! Today, weβre diving deep into the latest version of the Mistral 7B model, version 0.3! With a whopping 32,000 context window, improved tokenizer, and function calling support, this update promises significant advancements. π Mistral v3 Released! Did it Pass the Coding Test?
Massed compute: https://bit.ly/mervin-praison
Coupon: MervinPraison (50% Discount)
Connect to Massed Compute after Deploy: https://www.cendio.com/thinlinc/download/
In this video, weβll:
Compare Mistral 7B v0.3 with Llama 38B across various benchmarks π
Test Mistralβs coding abilities with Python challenges π
Evaluate its logical and reasoning skills π§
Assess safety features and function calling capabilities π
π§ Setup Guide:
Clone the Repository: git clone https://github.com/oobabooga/text-generation-webui
Navigate to Folder: cd text-generation-webui
Export Hugging Face Token: export HUGGINGFACE_TOKEN=your_token
Start Installation: bash start_linux.sh
Load the Model: Enter the model name and grant access.
π Links:
Patreon: https://patreon.com/MervinPraison
Ko-fi: https://ko-fi.com/mervinpraison
Discord: https://discord.gg/nNZu5gGT59
Twitter / X : https://twitter.com/mervinpraison
Sponsor a Video or Do a Demo of Your Product: https://mer.vin/contact/
https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3
π Weβll be using the unquantized version directly from Hugging Face, and testing its capabilities in various scenarios. Stay tuned as we push this model to its limits!
π If you find this video helpful, donβt forget to like, share, and subscribe for more AI content! Hit the bell icon π to stay updated.
π Timestamps:
0:00 β Introduction to Mistral 7B v0.3
0:11 β Comparison with Llama 38B
0:33 β Setup and Configuration
2:30 β Coding Ability Tests
5:32 β Logical & Reasoning Skills
7:16 β Safety Test
8:00 β Function Calling Demonstration
10:00 β Final Thoughts and Conclusion
by Mervin Praison
windows server dns
Very pleased you used Praison AI in this test
Perfect
Great video! liked and shared (already subscribed). But what is the best function calling model right now? Do you have a web or index with the ranking of your models you review? thank you!
thank you so much this is really informative
π― Key Takeaways for quick navigation:
00:00 π Mistral 7B version 0.3 released with 32,000 context window and better tokenizer, supporting function calling.
00:14 π Llama 38B instruct slightly outperforms Mistral 7B v0.3 in Medical QA accuracy.
00:24 π Comparison of Mistral versions 1, 2, and 3 performance, with v0.3 slightly better at coding than v0.2.
00:38 π€ Mistral 7B v0.3 and Llama 38B are equally competitive in performance.
01:06 π» Testing Mistral 7B v0.3 for coding ability, reasoning skills, and accessing the model on text generation web UI.
01:33 π οΈ Setting up the environment and loading the unquantized Mistral 7B v0.3 model.
03:25 π§ͺ Coding test: Mistral 7B v0.3 performed well on easy challenges but struggled with medium, hard, and expert challenges.
06:08 π§ Logical and reasoning test: Mistral 7B v0.3 answered some questions correctly but failed on others.
07:16 π Safety test: Mistral 7B v0.3 avoided providing information on illegal activities, showing some level of safety.
08:12 π°οΈ Function calling test using PraisAI tool: Mistral 7B v0.3 demonstrated function calling ability with Crew AI framework but struggled with Autogen.
Made with HARPA AI
Itβs a good start. Maybe parsed function calling works every time.
logic and reasoning data sets??? whats that?
Good tests, thanks!
Finally, function calling on open source model!