A5000015824- Distributed and Parallel Computing Systems
- Scientific Computing and Data Management
- Advanced Data Storage Technologies
- Research Data Management Practices
- Parallel Computing and Optimization Techniques
Deutsches Elektronen-Synchrotron DESY
2021
Jupyter notebooks are executable documents that displayed in a web browser. The notebook elements consist of human-authored contextual and computer code, computer-generated output from executing the code. Such outputs can include tables plots. be executed interactively, whole saved, re-loaded re-executed, or converted to read-only formats such as HTML, LaTeX PDF. Exploiting these characteristics, used improve effectiveness computational data exploration, documentation, communication,...
Abstract Batch scheduling systems are usually designed to maximise fair resource utilisation and efficiency, but less well for demanding interactive processing, which requires fast access resources while low upstart latency is only of secondary significance high throughput performance computing systems. The clusters at DESY intended as batch end users run massive analysis simulation jobs enabling turnaround systems, in particular when processing expected feed back operation instruments near...
DESY is one of the largest accelerator laboratories in Europe. It develops and operates state art accelerators for fundamental science areas high energy physics, photon development. While decades physics (HEP) has been most prominent user compute, storage network infrastructure, various scientific as with photons development have caught up are now dominating demands on infrastructure resources, significant consequences IT resource provisioning. In this contribution, we will present an...
Within WLCG, the DESY site in Hamburg is one of largest Tier-2 sites with about 18500 CPU cores for Grid workloads. Additionally, 8000 are available interactive user analyses National Analysis Factory [NAF]. After migrating these two batch systems onto a common HTCondor based set-up during previous four years, we recapitulate lessons learned transition especially since both use cases differ their For jobs start-up latencies negligible and primary focus on an optimal utilization resources....