Dredging a data lake

• DOI: 10.1145/3366624.3368170
• Publication Date: 2019-11-27T13:23:09Z
• Authors: Tyler J. Skluzacek
• Keywords: Dredging

The rapid generation of data from distributed IoT devices, scientific instruments, and compute clusters presents unique management challenges. influx large, heterogeneous, complex causes repositories to become siloed or generally unsearchable---both problems not currently well-addressed by file systems. In this work, we propose Xtract, a serverless middleware extract metadata files spread across heterogeneous edge computing resources. my future intend study how Xtract can automatically construct extraction workflows subject users' cost, time, security, allocation constraints. To end, will enable the creation searchable centralized index collections.