Efficient Data Systems for Scalable Analysis with Large Language Models