Curated for content, computing, and digital experience professionals

Day: August 14, 2020

Google open-sources LIT for evaluating natural language models

Google-affiliated researchers released the Language Interpretability Tool (LIT), an open source, framework-agnostic platform and API for visualizing, understanding, and auditing natural language processing models. It focuses on questions about AI model behavior, like why models made certain predictions and why they’re performing poorly with input corpora. LIT incorporates aggregate analysis into a browser-based interface that’s designed to enable explorations of text generation behavior. The tool set is architected so that users can hop between visualizations and analysis to test hypotheses and validate those hypotheses over a data set. New data points can be added on the fly and their effect on the model visualized immediately, while side-by-side comparison allows for two models or two data points to be visualized simultaneously. And LIT calculates and displays metrics for entire data sets to spotlight patterns in model performance, including the current selection, manually generated subsets, and automatically generated subsets.

LIT works with any model that can run from Python, the Google researchers say, including TensorFlow, PyTorch, and remote models on a server. And it has a low barrier to entry, with only a small amount of code needed to add models and data. The team cautions that LIT doesn’t scale well to large corpora and that it’s not “directly” useful for training-time model monitoring. But they say that in the near future, the tool set will gain features like counterfactual generation plugins, additional metrics and visualizations for sequence and structured output types, and a greater ability to customize the UI for different applications.

H/T VentureBeat:

Savan Group delivers cloud-Based AI and machine learning capability

Savan Group announced that it has partnered with Amazon Web Services (AWS) to establish a cloud-based artificial intelligence (AI) and machine learning (ML) platform. Savan Group applies AI solutions to address the data and information challenges of the Federal Government. Much of this data is is unstructured data locked away in documents, videos, audio, images, and paper. Using ML and natural language processing (NLP), Savan Group is analyzing and extracting untapped potential, turning data into information and information into knowledge to help government agencies increase the value of data for mission, service, and public good. Savan Group is now developing ML models with near unlimited scale using distributed cloud storage and GPU compute within a FedRAMP-authorized environment.

© 2020 The Gilbane Advisor

Theme by Anders NorenUp ↑