Adaptation Of Visual Models With Cross-Modal Regularization