Professional Skills
- Have knowledge of Python, C++, Java, and Rust programming.
- Be familiar with common data structures and algorithms.
- Be familiar with relevant knowledge of computer networks, operating systems, etc.
- Have an understanding of the usage of common tools and commands in the Linux environment.
- Have an understanding of basic machine learning algorithms such as support vector machines and decision trees.
- Have an understanding of convolutional neural networks, attention mechanisms, Transformer, etc.
- Have an understanding of the basic knowledge of building machine learning systems and have an understanding of basic CUDA programming.
- Have an understanding of LoRA model optimization techniques.
Internship Experiences
Company Name ***
- Docking of Flash attention with the CUDA backend and inference engine.
- Migration and development of Flash attention on the HIP side.
Company Name ***
- Study relevant model technologies such as Mamba.
- CUDA parallel programming and basic knowledge of traditional compilers.
Project Experiences
- MiniMind, an ultra-small language model with 25.8 million parameters, developed from scratch.
- Dataset cleaning and pre-training (Pretrain).
- Supervised fine-tuning (SFT) and LoRA fine-tuning.
- Shared Mixture of Experts (MoE).
Construction and Optimization of AI Development Framework for RISC-V Architecture
- Deploy a water safety detection system using the ncnn tool on the RISC-V development board.
- Use methods such as openmp, rvv, and yolov5-lite to optimize and accelerate the model’s inference speed and inference process.
DeepLearning Sys
- Study the CMU DL Systems course and learn simple GPU and CUDA programming.
- Build a simple deep learning system.
- Implement automatic differentiation.
- Implement the computational graph.
- Implement GPU matrix operation acceleration.
