Illinois Institute of Technology organized a Computer Science seminar on Thursday, January 30, 2025, that brought Valerie Hayot-Sasson to speak as a postdoctoral scholar at the University of Chicago. She also held a position at Argonne National Laboratory. The presentation by Hayot-Sasson occurred from 12:45 p.m. to 1:45 p.m. in Room 113 of Stuart Building and covered her research about data sharing through innovative computing frameworks and federated infrastructure.
Hayot-Sasson provided during her talk a list of current challenges that stand in the way of effective data management and transfer in federated computing systems. Modern scientific applications surpass traditional simulation-based practices through Artificial Intelligence (AI) and Machine Learning (ML) workflows because distributed execution capabilities are available in Ray, Dask, and Globus Compute frameworks. New frameworks introduce substantial added value by moving data management tasks to users, yet this creates problems in dealing with large-scale dataset handling and transfer.
The process of distributed computing has a recognized data identification problem at this seminar which results in temporal and referential decoupling across data streams. Any notion of data transfer between different infrastructure systems and geographical areas becomes impossible to envision. Practices that lead to such solutions increase both runtime duration and computational burdens via performance-reducing inefficiencies established throughout the methods. Effective data sharing becomes significantly harder because of security requirements linked to data security and access control and data integrity management.
The research author designed ProxyStore as a Python library which provides just-in-time object resolution for efficient federated communication. The design core of ProxyStore consists of Proxies which function as low-footprint references that point to serializable data managed by different connectors such as Redis and local file system and peer-to-peer group endpoints.
ProxyStore presents data communication patterns from scientific computing as high-level abstractions which include streaming and ownership together with futures. ProxyStore enables researchers and developers to boost execution time benefits while maximizing their CPU and GPU performance during scientific research. Results from various benchmarks demonstrate that this library has strong potential to become popular within the scientific community because of its outstanding performance enhancements.
ProxyStore delivers superior scalability and adaptability to accommodate present-day requirements that depend on large data processing. The deployment of ProxyStore within current workflow operations serves to connect data supplier and consumer entities while establishing optimal data transfers between distributed computing networks.
Hayot-Sasson finished her presentation by examining how efficient communication systems will develop in the future of federated computing applications. Open data initiatives stand as the primary force behind scientific innovation but the continuous streaming of data for scalable processing remains a challenge. Recent skyrocketing AI and ML workflow expansion necessitates the development of adaptive energy-efficient data-sharing solutions that users can easily handle.
The challenge needs resolution and she proposed an interdisciplinary solution for addressing it. Researchers from different fields including computer science and physics along with bioinformatics need to process enormous amounts of data so efficient flexible data-sharing solutions become an essential requirement. ProxyStore represents part of a wider initiative that brings forth intelligent automated data management tools that drive productivity enhancement and innovation growth.
The seminar delivered important insights to both students and teachers at Illinois Tech about developing data management techniques. There existed an opportunity to explore how emerging technologies particularly ProxyStore could integrate with academic research together with industrial implementation. Those who attended the seminar developed greater understanding about federated computing obstacles alongside potential emerging solutions.
More information about ProxyStore and its applications becomes accessible through Hayot-Sasson’s research conducted at the University of Chicago and Argonne National Laboratory. The progressive evolution of data-driven science depends on advanced tools such as ProxyStore to carry forward the future of computing. Scientific and engineering students together with researchers need to examine these technological developments while assessing their impact on different scientific fields and engineering domains.