Overview

Our VIP is unique in that teams of students can work on projects at all layers of the technical stack involved in research, from engineering the network, storage and compute compute, applications, as well as direct research projects.

High Speed Research Network

Projects in the network layer focus on developing the High Speed Research Network itself, including research and engineering of novel solutions to improve stability, visibility, and capabilities of the network.

Network Engineering Research Team

Skills: Router/Switch Design, Network Architecture, Automation, Ansible, Linux, Python

The Network Engineering Research team focuses on the implementation and research of cutting-edge network solutions that solve challenges specific to high speed networking. This includes advanced network designs, automation and development of Ansible plugins for emerging networking paradigms, network monitoring prototypes, development of capabilities w.r.t. the SONiC Network Operating System, and more.

Network Security Research Team

Skills: 802.1X, Network Architecture, Intrusion Detection, Python, Kubernetes, Zeek

The Network Security Research team focuses on research and engineering of security solutions for the HSRN, and Science DMZ networks in general. Notably, this involves building security solutions that specifically retain extreme performance characteristics of the network, including over 400 Gbps bandwidth or microsecond latencies. Projects may extend to campus networking security as well. Projects include engineering 802.1X Port Access Control, researching NIC-based firewalls or line-rate DoS, etc.

Network Data Science Team

Skills: PyTorch, Python, Prometheus, Statistics, Visualization

The Network Data Science team focuses on using data science and applied machine learning to solve problems related to network engineering and security of the network itself. This includes developing network intrusion detection systems leveraging A.I. that are deployed directly onto the HSRN itself, using A.I. to estimate quality of transmission of lightpaths over the optical transport network, as well as special projects.

Network Performance Engineering Team

Skills: Linux, Kernel Optimization, Hardware, NIC Programming, FPGA

The Network Performance Engineering team focuses on optimizing of network & system parameters to maximize throughput or lower latency of applications running on the HSRN. This involves testing various networking hardware (Intel, NVIDIA, Broadcom, etc) and software optimizations (Kernel parameters, Operating Systems, NUMA, NIC parameters, Ethernet Flow Control) over various experiment scenarios.

Research Media Storage

NYU Research Technology is prototyping a new research storage solution using Ceph / CephFS to function as a high throughput, inexpensive data archive. This storage solution can be mounted directly by users on the HSRN. This is a NSF-funded project awarded to NYU.

Storage Engineering Team

Skills: Distributed Filesystems, Ceph, Hardware, Linux

The Storage Engineering Team focuses on the hardware and software building and testing of the new equipment used in the storage cluster. This includes organization and sourcing of the equipment, fulfilling requirements and budget, hands-on building of the cluster, as well as software and client testing.

HSRN Kubernetes Cluster

NYU Research Technology operates a Kubernetes cluster that is directly connected to the HSRN, allowing researchers to run containers that can function as batch jobs, or long running services. One notable difference between Kubernetes and traditional HPC is that Kubernetes allows for services such as websites or APIs that can leverage the compute power of enterprise servers directly in research experiments.

Deployments Engineering Team

Skills: Kubernetes, Kyverno, Go

The Deployments Engineering team is involved in GitLab CI pipelines, deployments, and Kubernetes-related development in general. Deployments include anything from networking applications, websites, ML models, and much more. Their current focus is on node priority access implementation with Kyverno, dynamic cluster bursting with admiralty, node traffic logger via eBPF, and network configuration CI with BatFish. This team heavily interacts with other teams on the HSRN to leverage Kubernetes and containerized applications in specific domains, as well as advocate and assist in adoption of Kubernetes in research domains.

Corelink is a programmable low-latency messaging platform for real-time research applications. Corelink can be used to selectively broadcast messages between producers and consumers with low latency across a variety of platforms. Corelink allows easy organization and connection of streams by type, either manually or programmatically, and allows configuration of plugins to apply processing or filters to streams on the server.

Researchers use Corelink to move data between applications in real-time at very low latencies for specific applications, such as real-time concert audio transmission. Currently, we focus on developing and improving the main Corelink APIs, as well as creating plugins to different data sources/sinks that may enable new kinds of real-time research and applications previously not possible.

Corelink_Overview

Skills: C++, NodeJS, Python, Library Programming, Software Packaging

The Corelink Client Development team focuses on maintaining and developing of clients of various programming languages and interfaces, including NodeJS, Python, C++, C#, and more. They focus on development of the Corelink library and APIs, as well as usability and packaging.

Skills: Rust, NodeJS

The Corelink Server Development team focuses on implementing features and services on the Corelink server, including logging, performance tuning, and more. The team is also looking to rewrite the server in Rust.

Applications

Applications sit on top of the infrastructure maintained by Research Technology, and function as the layer which directly interact with researchers and research domains. These teams often collaborate directly with researchers to help solve specific problems, and increase efficiency of current researcher workflows by leveraging the various aspects of Research Technology.

Audio Engineering Team

Skills: C++, JUCE

The Audio Engineering team focuses on low latency transfer of audio over the HSRN. This team is responsible for maintaining and developing the Corelink Audio plugin as well as prototypes involving audio transfer. Much of the team’s work involves the JUCE framework, and the team has been involved in demonstrations such as a distributed concert between performers in New York and Arizona.

Web Applications Team

Skills: NodeJS, Databases, Frontend/Backend, Kubernetes

The Web Applications team focuses on development of key web applications, such as a researcher dashboard that allows for HSRN users to easily manage network resources, as well as special projects.

AR/VR Applications Team

Skills: C++, C#, Unity, Unreal

The AR/VR Aplications team focuses send, receive, and render mocap skeletal data on Unity, Unreal, and WebXR client through Corelink. With the Optitrack motion capture system at the Future Reality Lab, they are able to record and send motion capture data from a remote desktop, and receving data package in each client. They then decode it into boneID, position (vector3), and rotation (quaternion), then render the skeleton. Their next goal is to support sending/receiving/rendering multiple skeletons at the same time across all 3 platforms.

Real-time Applications Team

Skills: Python, Kubernetes, Message brokers, Domain-specific knowledge, sensors programming

The Real-time Applications team focuses on development and prototyping of real-time or “closed loop” research workflows. Using the HSRN’s low latency and high bandwidth to securely transfer data to a HPC edge machine, researchers can incorporate real-time data processing and AI into their experiments to exponentially increase research efficiency. This team may work directly with research labs and use cases supported by the HSRN.

VIP Management & Operations

Management of the VIP itself and ensuring operational excellence involves students in all projects, as well as a team dedicated to business and administrative functions.

Business, Finance and Administration Team

Skills: Project Management, Organizational Leadership, UI/UX, Marketing

The Business Finance and Administration team focuses on handling administrative, public facing, and recruitment aspects of the VIP, as well as logistical functions and development throughout the HSRN. This team is also involved with project management and leadership aspects of the program.

Team Lead Committee

Skills: Leadership

The Team Lead Committee consists of the team leads and mentors of each subteam, who meet weekly with other committees to discuss current progress and envision future projects. They play an important role in managing the VIP, which comprises approximately 40 people.

HSRN OSI Model