Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Apple Machine Learning Research
[[{“value”:”This paper has been accepted at the Data Problems for Foundation Models workshop at ICLR 2024. Large language models are trained on massive scrapes of the web, which are often unstructured, noisy, and poorly phrased. Current scaling laws show that learning from such data requires… Read More »Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Apple Machine Learning Research