A5068607680- Advanced Image and Video Retrieval Techniques
- Multimodal Machine Learning Applications
- Recommender Systems and Techniques
- Visual Attention and Saliency Detection
- Domain Adaptation and Few-Shot Learning
- Advanced Graph Neural Networks
- Image Retrieval and Classification Techniques
- Human Pose and Action Recognition
- Face recognition and analysis
- Ubiquitin and proteasome pathways
- Robotics and Sensor-Based Localization
- Glycosylation and Glycoproteins Research
- Advanced Bandit Algorithms Research
- Sentiment Analysis and Opinion Mining
- Caching and Content Delivery
- Generative Adversarial Networks and Image Synthesis
- Human Mobility and Location-Based Analysis
- Cancer-related Molecular Pathways
- Advanced Neural Network Applications
- Video Analysis and Summarization
- Complex Network Analysis Techniques
- Advanced Vision and Imaging
- Topic Modeling
- Video Surveillance and Tracking Methods
- COVID-19 diagnosis using AI
Princess Margaret Cancer Centre
2023-2024
University Health Network
2023-2024
University of Toronto
2022
University of California, Berkeley
2013-2018
Berkeley College
2016
We demonstrate that, with the availability of distributed computation platforms such as Amazon Web Services and open-source tools, it is possible for a small engineering team to build, launch maintain cost-effective, large-scale visual search system. also demonstrate, through comprehensive set live experiments at Pinterest, that content recommendation powered by improves user engagement. By sharing our implementation details learnings from launching commercial engine scratch, we hope becomes...
Deep metric learning aims to learn a function mapping image pixels embedding feature vectors that model the similarity between images. Two major applications of are content-based retrieval and face verification. For tasks, majority current state-of-the-art (SOTA) approaches triplet-based non-parametric training. verification however, recent SOTA have adopted classification-based parametric In this paper, we look into effectiveness classification based on datasets. We evaluate several...
Transformers have become the dominant model in natural language processing, owing to their ability pretrain on massive amounts of data, then transfer smaller, more specific tasks via fine-tuning. The Vision Transformer was first major attempt apply a pure transformer directly images as input, demonstrating that compared convolutional networks, transformer-based architectures can achieve competitive results benchmark classification tasks. However, computational complexity attention operator...
Over the past three years Pinterest has experimented with several visual search and recommendation systems, from enhancing existing products such as Related Pins (2014), to powering new Similar Looks (2015), Flashlight (2016), Lens (2017). This paper presents an overview of our discovery engine these services, shares rationales behind technical product decisions use object detection interactive user interfaces. We conclude that this significantly improves engagement in both tasks.
Heterotrimeric G proteins can be regulated by posttranslational modifications, including ubiquitylation. KCTD5, a pentameric substrate receptor protein consisting of an N-terminal BTB domain and C-terminal domain, engages CUL3 to form the central scaffold cullin-RING E3 ligase complex (CRL3 KCTD5 ) that ubiquitylates Gβγ reduces levels in cells. The cryo-EM structure 5:5:5 KCTD5/CUL3 NTD /Gβ 1 γ 2 assembly reveals highly dynamic with rotations over 60° between /CUL3 CTD /Gβγ moieties...
Graph convolutional networks (GCNs) are a powerful class of graph neural networks. Trained in semi-supervised end-to-end fashion, GCNs can learn to integrate node features and structures generate high-quality embeddings that be used for various downstream tasks like search recommendation. However, existing mostly work on homogeneous graphs consider single embedding each node, which do not sufficiently model the multi-facet nature complex interaction nodes real-world Here, we present...
Learned embeddings for products are an important building block web-scale e-commerce recommendation systems. At Pinterest, we build a single set of product called ItemSage to provide relevant recommendations in all shopping use cases including user, image and search based recommendations. This approach has led significant improvements engagement conversion metrics, while reducing both infrastructure maintenance cost. While most prior work focuses on from features coming modality, introduce...
At Pinterest, we utilize image embeddings throughout our search and recommendation systems to help users navigate through visual content by powering experiences like browsing of related searching for exact products shopping. In this work describe a multi-task deep metric learning system learn single unified embedding which can be used power multiple products. The solution present not only allows us train application objectives in neural network architecture, but takes advantage correlated...
We propose a system for user-aided visual localization of desert imagery without the use any metadata such as GPS readings, camera focal length, or field-of-view. The makes only publicly available digital elevation models (DEMs) to rapidly and accurately locate photographs in non-urban environments deserts. Our generates synthetic skyline views from DEM extracts stable concavity-based features these skylines form database. To localize queries, user manually traces on an input photograph. is...
Large-scale pretraining of visual representations has led to state-of-the-art performance on a range benchmark computer vision tasks, yet the benefits these techniques at extreme scale in complex production systems been relatively unexplored. We consider case popular discovery product, where are trained with multi-task learning, from use-case specific understanding (e.g. skin tone classification) general representation learning for all content embeddings retrieval). In this work, we describe...
Sequential models have become increasingly popular in powering personalized recommendation systems over the past several years. These approaches traditionally model a user's actions on website as sequence to predict next action. While theoretically simplistic, these are quite challenging deploy production, commonly requiring streaming infrastructure reflect latest user activity and potentially managing mutable data for encoding hidden state. Here we introduce PinnerFormer, representation...
Sequential models that encode user activity for next action prediction have become a popular design choice building web-scale personalized recommendation systems. Traditional methods of sequential either utilize end-to-end learning on realtime actions, or learn representations separately in an offline batch-generated manner. This paper (1) presents Pinterest's ranking architecture Homefeed, our product and the largest engagement surface; (2) proposes TransAct, model extracts users'...
As online content becomes ever more visual, the demand for searching by visual queries grows correspondingly stronger. Shop The Look is an shopping discovery service at Pinterest, leveraging search to enable users find and buy products within image. In this work, we provide a holistic view of how built Look, oriented system, along with lessons learned from addressing needs. We discuss topics including core technology across object detection embeddings, serving infrastructure realtime...
Abstract Heterotrimeric G proteins can be regulated by post-translational modifications, including ubiquitylation. KCTD5, a pentameric substrate receptor protein consisting of an N-terminal BTB domain and C-terminal (CTD), engages CUL3 to form the central scaffold cullin- RING E3 ligase complex (CRL3 KCTD5 ) that ubiquitylates Gβγ reduces levels in cells. The cryo-EM structure 5:5:5 KCTD5/CUL3 NTD /Gβ 1 γ 2 assembly reveals highly dynamic with rotations over 60° between /CUL3 CTD /Gβγ...
In this work, we present our journey to revolutionize the personalized recommendation engine through end-to-end learning from raw user actions. We encode user's long-term interest in PinnerFormer, a embedding optimized for future actions via new dense all-action loss, and capture short-term intention by directly real-time action sequences. conducted both offline online experiments validate performance of model architecture, also address challenge serving such complex using mixed CPU/GPU...
Graph Convolutional Networks (GCN) can efficiently integrate graph structure and node features to learn high-quality embeddings. At Pinterest, we have developed deployed PinSage, a data-efficient GCN that learns pin embeddings from the Pin-Board graph. Pinterest relies heavily on PinSage which in turn only leverages However, there exist several entities at heterogeneous interactions among these entities. These diverse provide important signal for recommendations modeling. In this work, show...
Putting together an ideal outfit is a process that involves creativity and style intuition. This makes it particularly difficult task to automate. Existing styling products generally involve human specialists highly curated set of fashion items. In this paper, we will describe how bootstrapped the Complete The Look (CTL) system at Pinterest. technology aims learn subjective "style compatibility" in order recommend complementary items complete outfit. particular, want show recommendations...
Sequential models have become increasingly popular in powering personalized recommendation systems over the past several years. These approaches traditionally model a user's actions on website as sequence to predict next action. While theoretically simplistic, these are quite challenging deploy production, commonly requiring streaming infrastructure reflect latest user activity and potentially managing mutable data for encoding hidden state. Here we introduce PinnerFormer, representation...
We present a memory scalable image localization system that uses distributed kd-trees created on overlapping geographic cells using database of 10 million Google Street View images for an area approximately 10,000 square kilometers in Taiwan. Given collection over region interest (ROI), we generate by dynamically creating are optimized so each cell contains roughly the same number images. then create from SIFT features extracted cell. When querying system, run traditional feature matching...
Over the past three years Pinterest has experimented with several visual search and recommendation services, including Related Pins (2014), Similar Looks (2015), Flashlight (2016) Lens (2017). This paper presents an overview of our discovery engine powering these shares rationales behind technical product decisions such as use object detection interactive user interfaces. We conclude that this significantly improves engagement in both tasks.
Graph Convolutional Networks (GCN) can efficiently integrate graph structure and node features to learn high-quality embeddings. These embeddings then be used for several tasks such as recommendation search. At Pinterest, we have developed deployed PinSage, a data-efficient GCN that learns pin from the Pin-Board graph. The contains board entities captures belongs interaction. However, there exist at Pinterest users, idea pins, creators, heterogeneous interactions among these add-to-cart,...