Juliana Freire

Juliana Freire

Institute Professor, Professor of Computer Science and Data Science

Posts by Collection

portfolio

publications

ArcheType: A Novel Framework for Open-Source Column Type Annotation Using Large Language Models

Published in Proceedings of the VLDB Endowment, 2024

Existing deep-learning approaches to semantic column type annotation (CTA) have important shortcomings: they rely on semantic types which are fixed at training time; require a large number of training samples per type; incur high run-time inference costs; and their performance can degrade when evaluated on novel datasets, even when types remain constant. Large language models have exhibited strong zero-shot classification performance on a wide range of tasks and in this paper we explore their use for CTA. We introduce ArcheType, a simple, practical method for context sampling, prompt serialization, model querying, and label remapping, which enables large language models to solve CTA problems in a fully zero-shot manner. We ablate each component of our method separately, and establish that improvements to context sampling and label remapping provide the most consistent gains. ArcheType establishes a new state-of-the-art performance on zero-shot CTA benchmarks (including three new domain-specific benchmarks which we release along with this paper), and when used in conjunction with classical CTA techniques, it outperforms a SOTA DoDuo model on the fine-tuned SOTAB benchmark.

Recommended citation: Benjamin Feuer, Yurong Liu, Chinmay Hegde, and Juliana Freire "ArcheType: A Novel Framework for Open-Source Column Type Annotation Using Large Language Models." Proceedings of the VLDB Endowment 2024
Download Paper

Sampling Methods for Inner Product Sketching

Published in Proceedings of the VLDB Endowment, 2024

Recently, Bessa et al. (PODS 2023) showed that sketches based on coordinated weighted sampling theoretically and empirically outperform popular linear sketching methods like Johnson-Lindentrauss projection and CountSketch for the ubiquitous problem of inner product estimation. We further develop this finding by introducing and analyzing two alternative sampling-based methods. In contrast to the computationally expensive algorithm in Bessa et al., our methods run in linear time (to compute the sketch) and perform better in practice, significantly beating linear sketching on a variety of tasks. For example, they provide state-of-the-art results for estimating the correlation between columns in unjoined tables, a problem that we show how to reduce to inner product estimation in a black-box way. While based on known sampling techniques (threshold and priority sampling) we introduce significant new theoretical analysis to prove approximation guarantees for our methods.

Recommended citation: Majid Daliri, Juliana Freire, Christopher Musco, Aécio Santos, and Haoxiang Zhang "Sampling Methods for Inner Product Sketching." Proceedings of the VLDB Endowment 2024
Download Paper

How can we make sound replication decisions?

Published in Proceedings of the National Academy of Sciences, 2025

Replication and the reported crises impacting many fields of research have become a focal point for the sciences. This has led to reforms in publishing, methodological design and reporting, and increased numbers of experimental replications coordinated across many laboratories. While replication is rightly considered an indispensable tool of science, financial resources and researchers’ time are quite limited. In this perspective, we examine different values and attitudes that scientists can consider when deciding whether to replicate a finding and how. We offer a conceptual framework for assessing the usefulness of various replication tools, such as preregistration.

Recommended citation: Clintin P. Davis-Stober, Alexandra Sarafoglou, Balazs Aczel, Suyog H. Chandramouli, Timothy M. Errington, Sarahanne M. Field, Ayelet Fishbach, Juliana Freire, John P. A. Ioannidis, Klaus Oberauer, Franco Pestilli, Susanne Ressl, Daniel J. Schad, Judith Ter Schure, Katya Tentori, Don Van Ravenzwaaij, Joachim Vandekerckhove, and Odd Erik Gundersen "How can we make sound replication decisions?." Proceedings of the National Academy of Sciences 2025
Download Paper

Magneto: Combining Small and Large Language Models for Schema Matching

Published in Proceedings of the VLDB Endowment, 2025

Recent advances in language models (LMs) open new opportunities for schema matching (SM). Recent approaches have shown their potential and key limitations: while small LMs (SLMs) require costly, difficult-to-obtain training data, large LMs (LLMs) demand significant computational resources and face context window constraints. We present Magneto, a cost-effective and accurate solution for SM that combines the advantages of SLMs and LLMs to address their limitations. By structuring the SM pipeline in two phases, retrieval and reranking, Magneto can use computationally efficient SLM-based strategies to derive candidate matches which can then be reranked by LLMs, thus making it possible to reduce runtime while improving matching accuracy. We propose (1) a self-supervised approach to fine-tune SLMs which uses LLMs to generate syntactically diverse training data, and (2) prompting strategies that are effective for reranking. We also introduce a new benchmark, developed in collaboration with domain experts, which includes real biomedical datasets and presents new challenges for SM methods. Through a detailed experimental evaluation, using both our new and existing benchmarks, we show that Magneto is scalable and attains high accuracy for datasets from different domains.

Recommended citation: Yurong Liu, Eduardo H. M. Pena, Aécio Santos, Eden Wu, and Juliana Freire "Magneto: Combining Small and Large Language Models for Schema Matching." Proceedings of the VLDB Endowment 2025
Download Paper

A Cost-Effective LLM-based Approach to Identify Wildlife Trafficking in Online Marketplaces

Published in Proceedings of the ACM on Management of Data, 2025

Wildlife trafficking remains a critical global issue, significantly impacting biodiversity, ecological stability, and public health. Despite efforts to combat this illicit trade, the rise of e-commerce platforms has made it easier to sell wildlife products, putting new pressure on wild populations of endangered and threatened species. The use of these platforms also opens a new opportunity: as criminals sell wildlife products online, they leave digital traces of their activity that can provide insights into trafficking activities as well as how they can be disrupted. The challenge lies in finding these traces. Online marketplaces publish ads for a plethora of products, and identifying ads for wildlife-related products is like finding a needle in a haystack. Learning classifiers can automate ad identification, but creating them requires costly, time-consuming data labeling that hinders support for diverse ads and research questions. This paper addresses a critical challenge in the data science pipeline for wildlife trafficking analytics: generating quality labeled data for classifiers that select relevant data. While large language models (LLMs) can directly label advertisements, doing so at scale is prohibitively expensive. We propose a cost-effective strategy that leverages LLMs to generate pseudo labels for a small sample of the data and uses these labels to create specialized classification models. Our novel method automatically gathers diverse and representative samples to be labeled while minimizing the labeling costs. Our experimental evaluation shows that our classifiers achieve up to 95% F1 score, outperforming LLMs at a lower cost. We present real use cases that demonstrate the effectiveness of our approach in enabling analyses of different aspects of wildlife trafficking.

Recommended citation: Juliana Silva Barbosa, Ulhas Gondhali, Gohar Petrossian, Kinshuk Sharma, Sunandan Chakraborty, Jennifer Jacquet, and Juliana Freire "A Cost-Effective LLM-based Approach to Identify Wildlife Trafficking in Online Marketplaces." Proceedings of the ACM on Management of Data 2025
Download Paper

Descriptive Analysis of Online Wildlife Products Using Vision Language Models

Published in Proceedings of the ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies, 2025

View publication

Recommended citation: Kinshuk Sharma, Juliana Silva Barbosa, Spencer Roberts, Ulhas Gondhali, Gohar Petrossian, Jennifer Jacquet, Juliana Freire, and Sunandan Chakraborty "Descriptive Analysis of Online Wildlife Products Using Vision Language Models." Proceedings of the ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies 2025
Download Paper

Bridging Disciplines in Data Management Research to Solve Complex Data Problems

Published in Proceedings of the VLDB Endowment, 2025

Scientific discovery has undergone profound transformations across multiple paradigms, each bringing new data challenges whose solutions demand bridging multiple areas of computer science. This talk presents a research journey spanning three scientific paradigms and projects that illustrate how domain-driven problems reveal fundamental data management challenges and drive interdisciplinary innovation. From the need to manage complex pipelines and their provenance in computational science (3rd paradigm), to new requirements that arise in data-driven discovery (4th paradigm) to support visual exploration of large-scale spatio-temporal data, and today’s AI-powered discovery paradigm (5th paradigm), where AI enables effective and general approaches to the long-standing data integration problem. The projects share a common pattern: complex scientific challenges demand more than single-discipline solutions, and by embracing collaboration across computer science areas and working closely with domain experts, we can identify fundamental research opportunities that lead to both methodological advances and systems with real-world impact.

Recommended citation: Juliana Freire "Bridging Disciplines in Data Management Research to Solve Complex Data Problems." Proceedings of the VLDB Endowment 2025
Download Paper

BDIViz: An Interactive Visualization System for Biomedical Schema Matching with LLM-Powered Validation

Published in IEEE Transactions on Visualization and Computer Graphics, 2026

View publication

Recommended citation: Eden Wu, Dishita G Turakhia, Guande Wu, Christos Koutras, Sarah Keegan, Wenke Liu, Beata Szeitz, David Fenyo, Cláudio T. Silva, and Juliana Freire "BDIViz: An Interactive Visualization System for Biomedical Schema Matching with LLM-Powered Validation." IEEE Transactions on Visualization and Computer Graphics 2026
Download Paper

LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake

Published in Proceedings of the International Conference on Machine Learning (ICML), 2026

Recommended citation: Haonan Wang, Jiaxiang Liu, Yurong Liu, Austin Senna Wijaya, Tianle Zhou, Eden Wu, Yijia Chen, Wanting You, Reya Vir, Daniela Pinto Veizaga, Grace Fan, Yusen Zhang, Juliana Freire, and Eugene Wu "LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake." Proceedings of the International Conference on Machine Learning (ICML) 2026

From FAIR to CURE: guidelines for computational models of biological systems

Published in npj Systems Biology and Applications, 2026

Abstract Guidelines for managing scientific data have been established under the FAIR principles, requiring that data be Findable, Accessible, Interoperable, and Reusable. In many scientific disciplines, especially computational biology, both data and models are key to progress. For this reason, and recognizing that such models are a very special type of “data”, we argue that computational models, especially mechanistic models prevalent in medicine, physiology and systems biology, deserve a complementary set of guidelines. We propose the CURE principles, emphasizing that models should be Credible, Understandable, Reproducible, and Extensible. We delve into each principle, discussing verification, validation, and uncertainty quantification for model credibility; the clarity of model descriptions and annotations for understandability; adherence to standards and open science practices for reproducibility; and the use of open standards and modular code for extensibility and reuse. We outline recommended and baseline requirements for each aspect of CURE, aiming to enhance the impact and trustworthiness of computational models, particularly in biomedical applications where credibility is paramount. Our perspective underscores the need for a more disciplined approach to modeling, aligning with emerging trends such as Digital Twins and emphasizing the importance of data and modeling standards for interoperability and reuse. Finally, we emphasize that given the non-trivial effort required to implement the guidelines, the community should strive to automate as many of the guidelines as possible.

Recommended citation: Herbert M. Sauro, Eran Agmon, Michael L. Blinov, John H. Gennari, Joseph L. Hellerstein, Adel Heydarabadipour, Bartholomew E. Jardine, Elebeoba May, David P. Nickerson, Lucian P. Smith, Gary D. Bader, Frank T. Bergmann, Patrick M. Boyle, Andreas Dräger, James R. Faeder, Song Feng, Juliana Freire, Fabian Fröhlich, James A. Glazier, Thomas E. Gorochowski, Tomas Helikar, Henning Hermjakob, Stefan Hoops, Peter Hunter, Princess I. Imoukhuede, Sarah M. Keating, Matthias König, Reinhard Laubenbacher, Leslie M. Loew, Carlos F. Lopez, William W. Lytton, Rahuman S. Malik-Sheriff, Andrew McCulloch, Pedro Mendes, Lealem Mulugeta, Chris J. Myers, Jerry G. Myers Jr, Anna Niarakis, David D. Van Niekerk, Brett G. Olivier, Alexander A. Patrie, Ellen M. Quardokus, Nicole Radde, Johann M. Rohwer, Sven Sahle, James C. Schaff, Falk Schreiber, T. J. Sego, Janis Shin, Jacky L. Snoep, Rajanikanth Vadigepalli, H. Steven Wiley, Dagmar Waltemath, and Ion I. Moraru "From FAIR to CURE: guidelines for computational models of biological systems." npj Systems Biology and Applications 2026
Download Paper

AutoDDG: Automated Dataset Description Generation using Large Language Models

Published in Proceedings of the ACM on Management of Data, 2026

The proliferation of datasets across open data portals and enterprise data lakes presents an opportunity for deriving data-driven insights. Widely-used dataset search systems rely on keyword search over dataset metadata to support discovery. Therefore, when metadata is incomplete, missing, or inconsistent with dataset contents, findability is severely compromised. To address this limitation, we introduce AutoDDG, a framework that automatically generates textual descriptions of tabular data. By adopting a data-driven approach to summarize dataset contents and leveraging large language models (LLMs) to enrich summaries with semantic information and produce human-readable text, AutoDDG derives descriptions that are comprehensive, accurate, readable, and concise. A critical challenge in this problem is evaluating the effectiveness of description generation methods and assessing the quality of the generated descriptions. We propose a comprehensive evaluation methodology that combines retrieval, reference-based, and reference-free assessment, with human validation. Our experimental results using new benchmarks demonstrate that AutoDDG generates high-quality, accurate descriptions at scale, significantly improving dataset retrieval performance across diverse use cases. AutoDDG is available at https://github.com/VIDA-NYU/AutoDDG.

Recommended citation: Haoxiang Zhang, Yurong Liu, Aécio Santos, Wei-Lun (Allen) Hung, and Juliana Freire "AutoDDG: Automated Dataset Description Generation using Large Language Models." Proceedings of the ACM on Management of Data 2026
Download Paper

AgentTrails: Towards Trust and Reuse for Agentic Tasks

Published in ACM SIGMOD DashSys Workshop, 2026

LLM-powered agents increasingly tackle complex tasks by invoking tools, querying databases, executing code, and manipulating intermediate artifacts. These agents follow trajectories that are typically stored as chronological logs, obscuring the underlying dataflow – the dependencies between their actions and the artifacts they create and manipulate. This limits developers’ ability to understand the agents’ trails, compare executions, debug failures, and re-use the computations. We present AgentTrails, a prototype system for agent provenance and sensemaking. AgentTrails converts raw trajectories into structured provenance graphs, where tool calls are modeled as computational actions and inputs and outputs as data artifacts. The system supports the comparison of executions by placing multiple provenance graphs on a shared canvas and constructing a joined quotient graph that aligns recurring tools, artifacts, and dependency structures across trajectories. On top of this representation, AgentTrails supports pattern extraction, downstream analysis, and skill abstraction. We demonstrate AgentTrails on real-world agent trajectories, showing that it reveals hidden dependencies, aligns divergent executions, and surfaces recurring tool-use patterns beyond chronological logs.

Recommended citation: Eden Wu, Sonia Castelo, Yurong Liu, Cláudio T. Silva, and Juliana Freire "AgentTrails: Towards Trust and Reuse for Agentic Tasks." ACM SIGMOD DashSys Workshop 2026
Download Paper

MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery

Published in Proceedings of the VLDB Endowment, 2026

Join discovery is a core task in dataset search, enabling users to find columns that can be joined with a given query column. Early approaches focused on equi-joins, but data lakes and open-data repositories often contain columns whose values refer to the same entity but use different syntactic representations. To address this challenge, recent approaches discover semantically joinable columns but face a fundamental trade-off: methods that perform value-level comparisons accurately identify joinable columns but scale poorly to columns with high cardinality; column-level methods that encode an entire column into a single embedding are efficient but do not capture the fine-grained value alignment that determines whether a join is possible. We present MosaicJoin, a value-level semantic join discovery method that balances this trade-off. MosaicJoin achieves scalability through a novel sketching strategy that approximates the joinability of a column pair without having to compare all values. At query time, MosaicJoin scores each candidate sketch using a joinability score at a cost bounded by the sketch size, making retrieval efficient even for high-cardinality columns. A query subsampling operator further reduces online search time with provable accuracy guarantees, enabling robust retrieval for large query columns. Extensive experiments show that MosaicJoin outperforms previously published methods across all benchmarks while running up to 66 times faster than other value-level methods. MosaicJoin requires no training or fine-tuning, and it scales robustly to query columns containing up to 57K values and data lake columns containing up to 1M values.

Recommended citation: Grace Fan, Eden Wu, Majid Daliri, and Juliana Freire "MosaicJoin: Compact Semantic Sketches for Value-Level Join Discovery." Proceedings of the VLDB Endowment 2026
Download Paper

Interoperability in Healthcare: A Primer and New Frontiers

Published in Proceedings of the VLDB Endowment, 2026

Interoperability in healthcare remains difficult despite decades of standards and infrastructure. Although messaging protocols and standards enable systems to exchange data, they do not guarantee that clinical meaning is preserved across organisations, workflows, and cultural contexts. This persistent gap between where semantics are used in care and how they are exchanged continues to drive fragmentation, bespoke mappings, and costly manual integration. This tutorial offers a primer and a forward-looking agenda by framing interoperability as three generations. We first revisit the pre-interoperability era of proprietary systems and point-to-point integrations. We then cover the standards-based second generation, highlighting what it solved (e.g., connectivity) and what it left unresolved (e.g., semantic alignment). Finally, we outline an emerging third generation towards true interoperability, and conclude with research directions for building interoperability in the AI era.

Recommended citation: Andra Ionescu, Paris Carbone, Sebastiaan Meijer, Christos Koutras, Juliana Freire, Katja Hose, and Jayanth Raghothama "Interoperability in Healthcare: A Primer and New Frontiers." Proceedings of the VLDB Endowment 2026
Download Paper

Semantic Data Systems: From Data Management to Data Understanding

Published in Proceedings of the VLDB Endowment, 2026

For decades, data management systems have relied on syntactic, structural, and statistical representations of data, leaving semantic understanding largely to human users. This division has become increasingly problematic as users analyze data disconnected from their original collection context, and yet must determine whether datasets are relevant, compatible, and trustworthy. Large language models (LLMs) introduce a new computational capability: the ability to reason about the meaning of data and leverage accumulated background knowledge. I argue that semantic understanding should be viewed as a new systems primitive for data management, enabling a new class of semantic data systems in which understanding becomes a first-class computational capability that complements syntactic and statistical representations of data. Drawing on our recent work spanning dataset discovery, semantic type discovery, schema matching, data harmonization, and agentic workflows, I show that LLMs alone do not solve data management problems: the central challenge is no longer enabling machines to understand data, but designing systems that can represent, maintain, evaluate, and exploit that understanding. These systems should externalize semantic knowledge into reusable artifacts, combine semantic reasoning with algorithms through structured workflows, subject decisions to human validation, and expose specialized capabilities as composable primitives that agents can orchestrate. This perspective suggests a research agenda for semantic data systems that extends beyond data discovery and integration toward a broader transformation of data management.

Recommended citation: Juliana Freire "Semantic Data Systems: From Data Management to Data Understanding." Proceedings of the VLDB Endowment 2026
Download Paper

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.