Case Study Innovating Domain Adaptation Through Continual Pre Training And Model Merging
Source
Arcee AI editorial sitemap
Author
Arcee AI editorial sitemap
Date
Terms in this piece · Glossary
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
Why it matters
Domain adaptation by continued pretrainingThe first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.Full definition → degrades a model's general capability; pairing it with model merging is presented as the cheaper way to gain the domain without paying that forgetting cost.
