From 7d09b4337b20b44b8d05296568f104013583e074 Mon Sep 17 00:00:00 2001 From: sjaipuriyar Date: Mon, 26 Aug 2024 13:51:52 -0700 Subject: [PATCH] Update README.md adding section for if the user wants to add more models or modify current models --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index fbfa912..a7a80ab 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,10 @@ Granite 3b generation with Go code: - If you haven't as yet, also make sure to fork Phi-3: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct#sample-inference-code$0 and IBM Granite 3b: https://huggingface.co/ibm-granite/granite-3b-code-instruct#generation$0 from huggingface and change the `@app.route` in the testing.py file to match the locations where you host those models. # **Adding more models for testing** -in prog... +in prog.. +- Head over to huggingface's website -> models -> select the model of your choice -> git clone it to the same directory as testing.py -> add condition within try block of `generate_response()` function to recognize the model you're adding. +- Additionally you should add a tokenizer that recognizes the token for that model such as `tokenizer_phi = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")` for the Phi-3 model +- Finally add the model using `AutoModelForCausalLM.from_pretrained` similarly to the phi and granite models shown in testing.py Overall the Phi-3 model can be better used as a way to generate text and the Granite model might be better at developing code and generating software solutions \ No newline at end of file