File size: 3,535 Bytes
2c75be5 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 a68b8cb 7f5b650 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | ---
{}
---
# Model Card for GPT_2_CODE
-Goal is to create a small GPT2 python coder
# Table of Contents
- [Model Card for GPT_2_CODE](#model-card-for--model_id-)
- [Table of Contents](#table-of-contents)
- [Table of Contents](#table-of-contents-1)
- [Model Details](#model-details)
- [Model Description](#model-description)
- [Uses](#uses)
- [Direct Use](#direct-use)
- [Downstream Use [Optional]](#downstream-use-optional)
- [Out-of-Scope Use](#out-of-scope-use)
- [Bias, Risks, and Limitations](#bias-risks-and-limitations)
- [Recommendations](#recommendations)
- [Training Details](#training-details)
- [Training Data](#training-data)
- [Training Procedure](#training-procedure)
- [Preprocessing](#preprocessing)
- [Speeds, Sizes, Times](#speeds-sizes-times)
- [Evaluation](#evaluation)
- [Testing Data, Factors & Metrics](#testing-data-factors--metrics)
- [Testing Data](#testing-data)
- [Factors](#factors)
- [Metrics](#metrics)
- [Results](#results)
- [Model Examination](#model-examination)
- [Environmental Impact](#environmental-impact)
- [Technical Specifications [optional]](#technical-specifications-optional)
- [Model Architecture and Objective](#model-architecture-and-objective)
- [Compute Infrastructure](#compute-infrastructure)
- [Hardware](#hardware)
- [Software](#software)
- [Citation](#citation)
- [Glossary [optional]](#glossary-optional)
- [More Information [optional]](#more-information-optional)
- [Model Card Authors [optional]](#model-card-authors-optional)
- [Model Card Contact](#model-card-contact)
- [How to Get Started with the Model](#how-to-get-started-with-the-model)
# Model Details
## Model Description
WIP,Goal is to create a small GPT2 python coder
- **Developed by:** C, o, d, e, M, o, n, k, e, y
- **Shared by [Optional]:** More information needed
- **Model type:** Language model
- **Language(s) (NLP):** eng
- **License:** wtfpl
- **Parent Model:** More information needed
- **Resources for more information:** More information needed
- [GitHub Repo](None)
- [Associated Paper](None)
# Uses
coding assistant
## Direct Use
generate python code snippets
## Downstream Use [Optional]
semi auto coder
## Out-of-Scope Use
describe code
Keep Finetuning on question/python datasets
# Training Details
## Training Data
flytech/python-codes-25k
espejelomar/code_search_net_python_10000_examples
## Training Procedure
Train/Val/Scheduler
### Preprocessing
More information needed
### Speeds, Sizes, Times
Epochs 3
flytech/python-codes-25k (4600)
Training Loss: 0.4007
Validation Loss: 0.5526
Epochs 3
espejelomar/code_search_net_python_10000_examples (4800)
Training Loss: 1.5355
Validation Loss: 1.1723
# Evaluation
Manual comparison with base model
### Testing Data
flytech/python-codes-25k
espejelomar/code_search_net_python_10000_examples
### Factors
80/20 train/val
### Metrics
train/validate
lr scheduling
## Results
Better in python code generation as base gpt2-medium model
# Model Examination
More information needed
# Environmental Impact
- **Hardware Type:** CPU and Colab T4
- **Hours used:** 4
- **Cloud Provider:** Google Colab
- **Compute Region:** NL
## Model Architecture and Objective
gpt2
## Compute Infrastructure
More information needed
### Hardware
CPU and Colab T4
### Software
pytorch, custom python
# More Information [optional]
Experimental
# Model Card Authors [optional]
CodeMonkeyXL
# Model Card Contact
K00B404 huggingface
# How to Get Started with the Model
Use the code below to get started with the model.
|