EleutherAI
/

gpt-j-6b

Text Generation

Model card Files Files and versions

How to fine tune or train with our own data?

#15

by ram77gowri - opened Feb 18, 2023

Hi,

I am a data engineer and pretty new to AI models. I am interested in building something with my work confluence page information for our internal chatbot for work. The use case is instead of searching the whole confluence for some details, it should answer like chatGPT.

Here are my questions?

Can I do this with gpt-j-6b or suggest another model?
What are the steps to fine-tune? Could someone point out if there is any existing codebase to do that?

Appreciate any help in this regard.

Thanks,
Ram

•

edited Apr 2, 2023

This video provides some information. Let us know how you fare.

https://www.youtube.com/watch?v=efPrtcLdcdM

Some of the links referenced:

https://github.com/yk/gpt-4chan-public
His code

https://zenodo.org/record/3606810#.YpjGgexByDU
Data set

The model ( no longer available): https://huggingface.co/ykilcher/gpt-4chan

Hi @ram77gowri were you able to fine-tune the gptj model?

@hsuyab Not yet, Got side tracked with some other work. Getting back again.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment