| --- |
| datasets: |
| - Souha-BH/DetectingRiskyHealthBehaviorsInTikTokVideos |
| --- |
| # Model Card for Multimodal Risk Behavior Detection Model |
|
|
| ## Model Overview |
| The Multimodal Risk Behavior Detection Model is designed to detect risky health behaviors in TikTok videos. |
| By leveraging both visual and textual features from TikTok video content, the model can classify whether a video portrays risky health behaviors, such as smoking, alcohol consumption, or unhealthy eating habits. |
| The model integrates two pre-trained architectures: BERT for text feature extraction and ResNet50 for video frame analysis, combining their outputs to make predictions. |
|
|
| ## Training Data |
| The model was trained on the "Detecting Risky Health Behaviors in TikTok Videos" dataset: https://huggingface.co/datasets/Souha-BH/DetectingRiskyHealthBehaviorsInTikTokVideos. |
| This dataset includes video metadata, captions, and video clips, which are labeled as either risky or non-risky. The data was collected using the Apify TikTok Hashtag Scraper and annotated for risky health behaviors. |
| The model uses the text column (captions) and the corresponding video files from the dataset to extract text and visual features. |
|
|
| - Training-Validation-Test Split: The dataset was split into training, validation, and test sets using the "Split" column. |
| - Training set: Used to train the model's parameters. |
| - Validation set: Used to tune hyperparameters and avoid overfitting. |
| - Test set: Used to evaluate the final performance of the model. |
|
|
| ## Model Architecture |
| The Multimodal Risk Behavior Detection Model follows a multimodal approach that integrates both textual and visual modalities. |
|
|
| - Textual Features: Extracted using BERT (bert-base-uncased), with tokenized video captions passed through BERT's transformer layers. |
| - Visual Features: Extracted using ResNet50, where frames from each TikTok video are resized and processed to generate high-level visual embeddings. |
| - Feature Fusion: The embeddings from BERT and ResNet50 are concatenated and passed through a series of fully connected layers with ReLU activations and dropout regularization to prevent overfitting. |
| - Classification Layer: The final layer is a single-unit sigmoid layer that outputs a probability between 0 and 1, with 0.5 as the threshold for classification. |
|
|
| ### Training Procedure |
|
|
| - Loss Function: Binary Cross-Entropy Loss (BCE) was used to compute the error between predicted probabilities and true labels. |
| - Optimizer: Adam optimizer with a learning rate of 2e-5. |
| - Batch Size: 4 video samples per batch. |
| - Epochs: The model was trained for 5 epochs. |
| - Video Frame Limit: Each video was sampled for 10 frames to reduce computational overhead. |
| - Augmentation and Normalization: Frames were resized to 224x224 and normalized using ImageNet's mean and standard deviation. |
| - |
| ### Evaluation Metrics |
| The model was evaluated on the test set using the following metrics: |
|
|
| - Accuracy: Measures the proportion of correct predictions. |
| - Precision: Measures how many of the predicted "risky" videos were actually risky. |
| - Recall: Measures how many of the actual risky videos were correctly identified. |
| - F1 Score: The harmonic mean of precision and recall, balancing both metrics. |
| - ROC-AUC: Measures the area under the ROC curve, showing the model's ability to distinguish between risky and non-risky videos. |
|
|
| ### Model Performance |
| After training for 5 epochs, the model's performance on the test set was as follows: |
|
|
| - Accuracy: 63.33% |
| - Precision: 55.00% |
| - Recall: 84.62% |
| - F1 Score: 66.67% |
| - ROC-AUC: 65.84% |
|
|
| ### Usage |
|
|
| - Input: A TikTok video and its corresponding caption. |
| - Output: A probability score indicating the likelihood that the video depicts a risky health behavior. |
|
|
| ### Limitations |
| - Data Balance: If the dataset is imbalanced (more non-risky videos than risky ones), the model may struggle with precision. |
| - Contextual Understanding: The model relies heavily on textual captions. If a caption does not explicitly describe risky behavior, the model may underperform. |
| - Limited Frame Sampling: Only 10 frames per video are processed, which may miss important content, especially for longer videos. |
