Wanjia Fu

wanjia_fu@brown.edu

I am a Computer Science and Mathematics student at Brown University and a researcher in Brown Intelligent Robotics and Brown Interactive 3D Vision & Learning Lab.

I enjoy unleashing my creativity to innovate, as well as cooperating with people who share my passion to solve challenging problems with technology.

Outside of STEM, I love dancing, playing the drums, and writing novels.

If you'd like to chat, feel free to reach out to me on linkedin or through email!

Downlaod CV
Scroll down

Research

(CVPR 2025) GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities Jan 2024 - Nov 2024
Brown Interactive 3D Vision & Learning Lab
Rao Fu*, Dingxi Zhang*, Alex Jiang, Wanjia Fu, Austin Funk, Daniel Ritchie, Srinath Sridhar
Advisors: Srinath Sridhar, Daniel Ritchie
  • Synchronized the 30 FPS frames in the RGB videos across 51 cameras in the Brown Interactive Capture Systems for accurate 2D and 3D key points and MANO model fitting.
  • Generated and accumulated contact heatmap on the palms and backs of the left and right hand across bimanual hand-object and hand-hand activities of diverse scenes, such as boxing, massage, frying, and tea drinking.
  • Collected data for the GigaHands dataset, mostly focusing on makeup, cleaning, and clothes folding scenes, which consist of more than forty actions with text annotations; and scanned the objects for 3D reconstruction.
  • Tried to implement 3D Dynamic Gaussian Splatting on the multi-view camera data of scenes.

    • Vibration Haptics: Hand contact detection and localization with IMU

    Jan 2024 - Sep 2024
    Brown Interactive 3D Vision & Learning Lab
    Advisors: Srinath Sridhar, Krishna Murthy Jatavallabhula
  • Designed portable wrist hardware device containing with Inertial Measurement Unit (IMU), using which to collect hand object contact sensor and visual data
  • Preprocessed RGB camera data with MANO fitting pipeline to obtain pose estimation results and contact heatmap, built a neural network to improve hand-object contact detection and localization
  • Integrated IMU sensor and its data visualization into Brown Interaction Capture System (BRICS), participated in the hardware assembling and software calibration for the room-sized capture studio with professor Dr. Srinath Sridhar

    • Shaped-Based Skill Transfer by Learning Policy on Object Parts

    2023-2024
    Brown Intelligent Robotics Lab
    Advisor: George Konidaris
  • Learned latent representations of object parts for mugs and spatulas to carry out pouring and scooping tasks, and tried to learn a robust skill for all shapes of the same object category, with advisor Dr. George Konidaris
  • Incorporated Segment-Anything model to segment images of mugs into cup and handle and back projected the segmentation into point clouds.
  • Developed a policy with lower costs than when learning on the whole object instead of object parts
  • Worked on ROS and reinforcement learning, Boston Dynamics Spot robots, and KUKA robotic arms with radar cameras

    • Experience

    Intelligent Customer Service Product Operation Engineer Intern

    May - June 2023
    Hydsoft Technology Co., Ltd.
    Software Technology Outsourcing Services Company
  • Designed AI customer service to improve plane ticket booking system based on Baidu UNIT and natural language processing
  • Received the PaddlePaddle AI Technical Engineer Certificate
  • Received the Certificate of Achievement as Intelligent Customer Service Product Operation Engineer by Baidu AI Cloud
  • Designed front-end poster layout and user interface in a 10-member team for China Mobile Smart Card Production Platform

    • Software Engineering Intern at Brown Visual Computing

    June 2023 - August 2023
    Pinhole Camera Models in Computer Vision vs. Computer Graphics
    Javascript, Three.js, HTML
  • Employed front-end and back-end development with Three.js, Javascript, HTML, and CSS to create an online website tutorial on camera projection and perspective projection for the class Computer Vision for Dr. James Tompkin
  • Improved upon two existing online tutorials on affine transformations and the fundamental matrix for course development

    • Undergraduate Teaching Assistant

    Jan - May 2024
    Brown University CSCI 1430: Computer Vision
  • Work as one of the 24 undergraduate teaching assistants for the class CSCI 1430 Computer Vision
  • Engaged in course development, improved its webpage, Github repository, Gradescope autograders, and assignment code management
  • Graded and attended TA Hours and ED Hours for projects on image filtering, feature matching, camera geometry, scene classification, convolutional neural network, and a computer vision final project

    • Head Teaching Assistant

    Sep - Dec 2024
    Brown University CSCI 1430: Computer Vision
  • Work as one of the two head teaching assistants for the class CSCI 1430 Computer Vision
  • In addition to usual teaching assistant work, organize fellow ten undergraduate teaching assistants, directly help manage and develop the course with professor Dr. Srinath Sridhar, and host weekly grading/staff meetings

    • Projects

    MODEL BASED REINFORCEMENT LEARNING

    TOLD-ZERO: Generalize TD-MPC2 to discrete action spaces
    Jax, Gymnasium, Python
  • Integrate Monte Carlo Tree Search (MCTS) into learning based on Task-Oriented Latent Dynamics (TOLD) model.
  • Benchmark TD-MPC2 (Temporal Difference Model Predictive Control) on tasks with continuous action spaces using LightZero framework, and generalize TD-MPC2 to discrete action spaces.
  • Analyze the role of planning in model-based reinforcement learning (MBRL), including the Model Predictive Path Integral (MPPI).

    • COMPUTER GRAPHICS

    Interactive Ball, Particle System, Camp Scene
    OpenGL, Blender, C++, C
  • Generate fire particles that appear, rise, and disappear while changing color as flames in between rock and stick primitives.
  • Create a ball that can change in material, glow when it touches the fire, catch fire or steam in smoke according to its material, extinguish the smoke after it touches the water, and put out fire.
  • Create dynamic water primitive with randomized heights.
  • Construct camp scene using Blender which contains collision system with the ball.

    • STABLE DIFFUSION

    Text-to-Image Generation using Stable Diffusion
    Tensorflow, Python
  • Implemented image generation with tensorflow by training and testing on Fashion MNIST datset.
  • Achieved lower average loss than the original paper implemented in Pytorch.

    • IMAGE COLORIZATION

    U-Net, Deep CNN, and Conditional GAN for Auto-Colorization
    Pytorch, Python
  • Implement image auto-colorization using UNet, Conditional GAN (Generative Adversarial Network) and Deep-Koalarization to compare these three models and their performance on the same two datsets.
  • The patches of white in the images and the uneven distribution of colors in the ground-truth images might be the cause of ineffective auto-colorization.
  • Replaced the generator in Conditional GAN with our trained UNet to improve performance.

    • SOULFOOD

    Restaurant Recommendation WebApp with Blog Posting Functionality
    Figma, Firebase, Flask, React, Python
  • Built React WebApp that recommends restaurants based on labels input by the user using Figma and Python as backend
  • Designed and implemented front-end functionalities to allow users to post blogs about their experience and recommendation, as well as for other users to like, collect, or follow them.
  • Mocked data in the backend including user and post id, restaurant labels, and user reviews using Firebase

    • Education

    BROWN UNIVERSITY

    Providence, RI Sep 2022 - May 2026
    Sc.B. Computer Science; A.B. Mathematics
  • GPA: 4.0 / 4.0
  • Relevant Coursework (Computer Science):
  • Learning and Sequential Decision Making
  • Deep Learning
  • Computer Vision
  • Computer Graphics
  • Introduction to Software Engineering
  • Introduction to Computer Systems
  • Accelerated Introductino to Computer Science
  • Applied Statistics in Python
  • Individual Independent Study
  • Relevant Coursework (Mathematics):
  • Operations Research: Probabilistic Models
  • Abstract Algebra
  • Honors Linear Algebra
  • Applied Partial Differential Equations
  • Honors Multivariable Calculus
  • Honors Statistical Inference
  • Applied Ordinary Differential Equations
  • Other Coursework:
  • Intermediate Microeconomics (Mathematics)
  • Intermediate Macroeconomics
  • The Border / La Frontera
  • Leadership & Activities:
  • Brown Moli East Asian Dance Company: Co-Secretary Chair; Co-Captain
  • Organized members, logistics, and finance for the club, communicate with student activities office and financial supervisors
  • Prepare for annual spring show, teach Chinese dances, organize rehearsals, participate in shows on and off campus
  • Taught three dance pieces in the past year.
  • Brown IgniteCS: Member
  • Designed and developed a series of courses related to AI and taught as guest lecturer at Nathanael Greene Middle School
  • Designed and developed the coding club on HTML, CSS at the Sophia Academy coding club for 1.5 hours per week

    • SHANGHAI FOREIGN LANGUAGE SCHOOL

    Shanghai, China Sep 2019 - June 2022
    High School Diploma
  • GPA: 4.0 / 4.0
  • Leadership:
  • President of the Student Deaprtment of Arts and Sports
  • Vice President of the SFLS Physics Club
  • Vice President of the SFLS Mathematics Club
  • Vice President of the SFLS Acadeca Club

    • Skills

  • Technical Skills:
  • Expert: Python
  • Proficient: HTML, CSS
  • Fluent: Golang, Java, C++, Typescript, Javascript, WebGL, OpenGL, React
  • Prior Experience: C, Pyret, Racket
  • Language:
  • Trilingual proficiency in English, Spanish (Awarded C1 by El Instituto Cervantes), Chinese
  • Interests:
  • Semi-professional Chinese traditional dance, piano, taekwondo, novel writing, painting, drums, guitar, and tennis