In this paper, we consider distributed optimization problems where the goal is to minimize a sum of objective functions over multi-agent network. We focus on case when inter-agent communication described by strongly-connected, \emph{directed} graph. The proposed algorithm, ADD-OPT (Accelerated Distributed Directed Optimization), achieves best known convergence rate for class problems,~$O(\mu^{k}),0<\mu<1$, given strongly-convex, with globally Lipschitz-continuous gradients, where~$k$ number...

This paper considers a distributed optimization problem over multiagent network, in which the objective function is sum of individual cost functions at agents. We focus on case when communication between agents described by directed graph. Existing algorithms for graphs require least knowledge neighbors' outdegree each agent (due to requirement column-stochastic matrices). In contrast, our algorithm requires no such knowledge. Moreover, proposed achieves best known rate convergence this...

We propose Directed-Distributed Projected Subgradient (D-DPS) to solve a constrained optimization problem over multi-agent network, where the goal of agents is collectively minimize sum locally known convex functions. Each agent in network owns only its local objective function, commonly set. focus on circumstance when communications between are described by directed network. The D-DPS combines surplus consensus overcome asymmetry caused communication analysis shows convergence rate be O( ln k /√k ).

This paper develops a fast distributed algorithm, termed DEXTRA, to solve the optimization problem when n agents reach agreement and collaboratively minimize sum of their local objective functions over network, where communication between is described by directed graph. Existing algorithms restricted graphs with convergence √ rates O(ln k/ √k) for general convex k/k) are strongly convex, k number iterations. We show that, appropriate step-size, DEXTRA converges at linear rate O(τ <sup...

In this paper, we discuss distributed optimization over directed graphs, where doubly-stochastic weights cannot be constructed. Most of the existing algorithms overcome issue by applying push-sum consensus, which utilizes column-stochastic weights. The formulation requires each agent to know (at least) its out-degree, may impractical in e.g., broadcast-based communication protocols. contrast, describe FROST (Fast Row-stochastic-Optimization with uncoordinated STep-sizes), an algorithm...

This paper develops a fast distributed algorithm, termed \emph{DEXTRA}, to solve the optimization problem when~$n$ agents reach agreement and collaboratively minimize sum of their local objective functions over network, where communication between is described by a~\emph{directed} graph. Existing algorithms restricted directed graphs with convergence rates $O(\ln k/\sqrt{k})$ for general convex k/k)$ when are strongly-convex, where~$k$ number iterations. We show that, appropriate step-size,...

Recent advancements in large language models (LLMs) have spurred growing interest automatic theorem proving using Lean4, where effective tree search methods are crucial for navigating proof spaces. While the existing approaches primarily rely on value functions and Monte Carlo Tree Search (MCTS), potential of simpler like Best-First (BFS) remains underexplored. This paper investigates whether BFS can achieve competitive performance large-scale tasks. We present \texttt{BFS-Prover}, a...

This paper considers distributed convex optimization problems over a multi-agent network, with each agent possessing dynamic objective function. The agents aim to collectively track the minimum of sum locally known time-varying functions by exchanging information between neighbors. We focus on scenarios when communication among is described directed network. devise an algorithm discrete time-sampling scheme such that distance any estimate and optimal solutions converges steady state error...

In recent advancements in Conversational Large Language Models (LLMs), a concerning trend has emerged, showing that many new base LLMs experience knowledge reduction their foundational capabilities following Supervised Fine-Tuning (SFT). This process often leads to issues such as forgetting or decrease the model's abilities. Moreover, fine-tuned models struggle align with user preferences, inadvertently increasing generation of toxic outputs when specifically prompted. To overcome these...

Distributed Gradient Descent (DGD) is a well established algorithm to solve the minimization of sum multi-agents' objective functions in network, with assumption that network undirected, i.e., requiring weight matrices be doubly-stochastic. In this paper, we present distributed algorithm, called Directed-Distributed (D-DGD), same problem over directed graphs. each iteration D-DGD, augment an additional variable at agent record change state evolution. The simultaneously constructs...

In this paper, we propose Distributed Mirror Descent (DMD) algorithm for constrained convex optimization problems on a (strongly-)connected multi-agent network. We assume that each agent has private objective function and constraint set. The proposed DMD employs locally designed Bregman distance at agent, thus can be viewed as generalization of the well-known Projected Subgradient (DPS) methods, which use identical Euclidean distances agents. At iteration DMD, optimizes its own adjusted with...

We develop a fast distributed algorithm, termed DEXTRA, to solve optimization problems when N agents reach agreement and collaboratively minimize the sum of their local objectives over network, where communications between are described by directed graph. Existing algorithms, including Gradient-Push (GP) Directed-Distributed Gradient Descent (D-DGD), this problem restricted graphs with convergence rate O(ln k/√k). Our analysis shows that DEXTRA converges at linear O(ϵk) for some constant ϵ <...

The click-through rate (CTR) prediction task is to predict whether a user will click on the recommended item. As mind-boggling amounts of data are produced online daily, accelerating CTR model training critical ensuring an up-to-date and reducing cost. One approach increase speed apply large batch training. However, as shown in computer vision natural language processing tasks, with easily suffers from loss accuracy. Our experiments show that previous scaling rules fail neural networks. To...

Visible light communication (VLC) technology enabling data transmission by modulating solid-state light-emitting diode (LED) devices has attracted much attention recently. However, the rate of VLC remains low due to bandwidth performance LEDs. We present a transmitter consisting an LED array enable implementation 16-level pulse amplitude modulation scheme, which theoretically increases from 10Mbps 40Mbps. In addition, provides sufficient brightness for illumination office-sized locations....

In this paper, we propose a distributed algorithm, called Directed-Distributed Gradient Descent (D-DGD), to solve multi-agent optimization problems over directed graphs. Existing algorithms mostly deal with similar under the assumption of undirected networks, i.e., requiring weight matrices be doubly-stochastic. The row-stochasticity matrix guarantees that all agents reach consensus, while column-stochasticity ensures each agent's local gradient contributes equally global objective. graph,...

We present a failure recovery framework for System S, large-scale stream data analysis environment. It is intended to support multiple sites, which have their own local administration and goals. However, it beneficial these sites cooperate with each other, especially in the presence of various failures. Our ultimate goal automatic, timely through cooperation among sites. identify unique challenges context S our initial design work. In particular, we consider backup selection problem,...

We propose a distributed algorithm, termed the Directed-Distributed Projected Subgradient (D-DPS), to solve constrained optimization problem over multi-agent network, where goal of agents is collectively minimize sum locally known convex functions. Each agent in network owns only its local objective function, commonly set. focus on circumstance when communications between are described by directed network. The D-DPS augments an additional variable for each agent, overcome asymmetry caused...

This paper considers a distributed optimization problem over multi-agent network, in which the objective function is sum of individual cost functions at agents. We focus on case when communication between agents described by \emph{directed} graph. Existing algorithms for directed graphs require least knowledge neighbors' out-degree each agent (due to requirement column-stochastic matrices). In contrast, our algorithm requires no such knowledge. Moreover, proposed achieves best known rate...

With the rapid advancement of large language models (LLMs), there is a pressing need for comprehensive evaluation suite to assess their capabilities and limitations. Existing LLM leaderboards often reference scores reported in other papers without consistent settings prompts, which may inadvertently encourage cherry-picking favored prompts better results. In this work, we introduce GPT-Fathom, an open-source reproducible built on top OpenAI Evals. We systematically evaluate 10+ leading LLMs...

The emergence of Large Language Models (LLMs) such as ChatGPT and LLaMA encounter limitations in domain-specific tasks, with these models often lacking depth accuracy specialized areas, exhibiting a decrease general capabilities when fine-tuned, particularly analysis ability small sized models. To address gaps, we introduce ICE-GRT, utilizing Reinforcement Learning from Human Feedback (RLHF) grounded Proximal Policy Optimization (PPO), demonstrating remarkable in-domain scenarios without...

Recent developments in visible light communication (VLC) technology have solidified utilizing light-emitting diodes (LEDs) for not only illumination, but also optical wireless communication. This paper presents a novel access point transceiver that features addressable arrays of LEDs and photodetectors. The transmitter array enables combined illumination control serial data transmission an 16 producing aggregate rate 100 Mb/s. receiver consists 16-element broadband channels. Designed 0.5 μm...