There is a tendency across different subfields in AI to valorize small collection of influential benchmarks. These benchmarks operate as stand-ins for range anointed common problems that are frequently framed foundational milestones on the path towards flexible and generalizable systems. State-of-the-art performance these widely understood indicative progress long-term goals. In this position paper, we explore limits such order reveal construct validity issues their framing functionally...

Many approaches in biomedical informatics (BMI) rely on the ability to define, gather, and manipulate data support health through a cyclical research-practice lifecycle. Researchers within this field are often fortunate work closely with healthcare public systems influence generation capture have access vast amount of data. informaticists also expertise engage stakeholders, develop new methods applications, policy. However, research policy that explicitly seeks address systemic drivers would...

Racial discrimination has been a central driver of residential segregation for many decades, in the Seattle area as well United States whole. In addition to redlining and restrictive housing covenants, advertisements included explicit racial language until 1968. Since then, patterns have remained racialized, despite overt forms becoming less prevalent. this paper, we use Structural Topic Models (STM) qualitative analysis investigate how contemporary rental listings from Seattle-Tacoma

Previous work on classifying Twitter users’ political alignment has mainly focused lexical and social network features. This study provides evidence that affiliation is also reflected in features which have been previously overlooked: discourse patterns (proportion of Tweets are retweets or replies) their rate use capitalization punctuation. We find robust differences between politically left- right-leaning communities with respect to these sub-lexical features, although they not enough...

Inferring the nature of relationships between biomedical entities from text is an important problem due to difficulty maintaining human-curated knowledge bases in rapidly evolving fields. Neural word embeddings have earned attention for apparent ability encode relational information. However, embedding models that disregard syntax during training are limited their structural fundamental cognitive theories analogy. In this paper, we demonstrate utility encoding dependency structure a model...

We propose using a multilabel probing task to assess the morphosyntactic representations of multilingual word embeddings. This tweak on canonical makes it easy explore representations, both holistically and at level individual features (e.g., gender, number, case), leads more naturally study how language models handle co-occurring agreement phenomena). demonstrate this with BERT (Devlin et al., 2018), training probes for seven typologically diverse languages: Afrikaans, Croatian, Finnish,...

With the growth of Automatic Content Moderation (ACM) on widely used social media platforms, transparency into design moderation technology and policy is necessary for online communities to advocate themselves when harms occur.In this work, we describe a suite interactive modules support exploration various aspects technology, particularly those components that rely English models datasets hate speech detection, subtask within ACM. We intend demo stakeholders ACM in investigating definitions...

Many datasets contain personally identifiable information, or PII, which poses privacy risks to individuals. PII masking is commonly used redact personal information such as names, addresses, and phone numbers from text data. Most modern pipelines involve machine learning algorithms. However, these systems may vary in performance, that individuals particular demographic groups bear a higher risk for having their exposed. In this paper, we evaluate the performance of three off-the-shelf on...

Many datasets contain personally identifiable information, or PII, which poses privacy risks to individuals. PII masking is commonly used redact personal information such as names, addresses, and phone numbers from text data. Most modern pipelines involve machine learning algorithms. However, these systems may vary in performance, that individuals particular demographic groups bear a higher risk for having their exposed. In this paper, we evaluate the performance of three off-the-shelf on...

We introduce a multilabel probing task to assess the morphosyntactic representations of word embeddings from multilingual language models. demonstrate this with BERT (Devlin et al., 2018), training probes for seven typologically diverse languages varying morphological complexity: Afrikaans, Croatian, Finnish, Hebrew, Korean, Spanish, and Turkish. Through simple but robust paradigm, we show that renders many features easily simultaneously extractable (e.g., gender, grammatical case,...