sjaipuriyar/phi3_model_testing
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README.md

Browse source 
adding section for if the user wants to add more models or modify current models
This commit is contained in:
sjaipuriyar 2024-08-26 13:51:52 -07:00
parent b3f3cecfaa
commit 7d09b4337b
1 changed files with 4 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
README.md
View file

@ -30,7 +30,10 @@ Granite 3b generation with Go code:
- If you haven't as yet, also make sure to fork Phi-3: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct#sample-inference-code$0 and IBM Granite 3b: https://huggingface.co/ibm-granite/granite-3b-code-instruct#generation$0 from huggingface and change the `@app.route` in the testing.py file to match the locations where you host those models. - If you haven't as yet, also make sure to fork Phi-3: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct#sample-inference-code$0 and IBM Granite 3b: https://huggingface.co/ibm-granite/granite-3b-code-instruct#generation$0 from huggingface and change the `@app.route` in the testing.py file to match the locations where you host those models.
# **Adding more models for testing** # **Adding more models for testing**
in prog... in prog..
- Head over to huggingface's website -> models -> select the model of your choice -> git clone it to the same directory as testing.py -> add condition within try block of `generate_response()` function to recognize the model you're adding.
- Additionally you should add a tokenizer that recognizes the token for that model such as `tokenizer_phi = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")` for the Phi-3 model
- Finally add the model using `AutoModelForCausalLM.from_pretrained` similarly to the phi and granite models shown in testing.py
Overall the Phi-3 model can be better used as a way to generate text and the Granite model might be better at developing code and generating software solutions Overall the Phi-3 model can be better used as a way to generate text and the Granite model might be better at developing code and generating software solutions