This model is training on a HPC platform using 5 GeForce RTX 3090. Resource usage summary:

CPU time :                                   150541.41 sec.
Max Memory :                                 11148 MB
Average Memory :                             8411.42 MB
Total Requested Memory :                     -
Delta Memory :                               -
Max Swap :                                   -
Max Processes :                              4
Max Threads :                                195
Run time :                                   67735 sec.
Turnaround time :                            67774 sec.

Early stopping due to no improvement in validation loss after 7 epoch. Accuracy: 0.9709595558622203

<aside> 📢 More about ‘Chinese BERT with Whole Word Masking’

</aside>

The finetune model is based on hfl/chinese-bert-wwm:

https://github.com/ymcui/Chinese-BERT-wwm

• Whole Word Masking (wwm) is an upgraded version by BERT released on late May 2019.
• The Whole Word Masking (wwm) model can be easily get access at:

hfl/chinese-bert-wwm · Hugging Face

We explain the steps we take to finetune the pre-trained chinese-bert-wwm:

To classify job postings to the correct SOC occupation, we finetune a multi-class flat classification model based on a BERT model. We spotlight some of the key elements below. The full training code:

Job-Posting/flat_classification.ipynb at main · lzxlll/Job-Posting

1. Split the final estimation data into train, validation, and test sets as follows:

• The initial train_test_split call splits df into train_df_sample (60% of the data) and temp_df_sample (40% of the data).
• The second train_test_split call further splits temp_df_sample into valid_df_sample (50% of temp_df_sample, or 20% of the original data) and test_df_sample (50% of temp_df_sample, or 20% of the original data).