#Algorithm

OCW

  1. Hello 演算法

#Software

Meaning of Abbreviations

Shortcut Description
PR Pull Request
PTAL Please Take A Look
TBR To Be Reviewed
TL;DR Too Long; Didn’t Read
LGTM Looks Good To Me
SGTM Sounds Good To Me
TBD To Be Done

Environment

Linux

Docker

Installation
  1. 適用於Ubuntu與WSL的Nvidia Docker安裝方式
  2. Tensorflow with GPU on Windows WSL using Docker
  3. 【技術分享】重灌後，安裝 GPU 驅動以及 Docker
docker network-alias
  1. 談談 docker network-alias
docker stopsignal
Book
  1. 不可不知的 Docker 開發部署實戰筆記：網站工程師一定要會的 8大核心能力
Error
  1. Docker Desktop WSL ext4.vhdx too large
  2. 設定 super user - nvidia-docker :权限被拒绝
  3. Docker image platform compatibility issue with MAC Silicon processors(M1/M2)
  4. Solution for docker: invalid reference format: repository name must be lowercase

    Python

Design Pattern

基本
State Design Pattern
  1. The State Design Pattern in Python Explained
Class
  1. Python dataclass 教學：輕鬆定義資料類別
  2. Python Property 教學：保護變數資料的 Getter 與 Setter
  3. Python Type Hints 教學：我犯過的 3 個菜鳥錯誤
  4. Pydantic: Simplifying Data Validation in Python
Enum
  1. Python Enum 枚舉教學：有條理管理常數集合的好工具
Documentation
  1. mkdocstrings/griffe
  2. Top 6 Tools to Turn Code into Beautiful Diagrams
  3. Diagrams as Code
Project
  1. astral-sh/uv

    An extremely fast Python package installer and resolver, written in Rust.

  2. astral-sh/rye

    a Hassle-Free Python Experience

  3. cookiecutter

    A cross-platform command-line utility that creates projects from cookiecutters (project templates), e.g. Python package projects, C projects.

  4. pypa/hatch

    Modern, extensible Python project management

Flow Chart
  1. cdfmlr/pyflowchart
  2. PYTHON 自動產生 FLOWCHART !
Pipeline
  1. Data PIPELINE - Orchest
  2. airflow使用指南-机器学习工程自动化

#Python

#Julia

#CI/CD

Version Control

git
DVC

ML Experiments Management with Git

  1. Data Version Control | Git for Data & Models | ML Experiments Management
  2. Data Version Control With Python and DVC
elementary

Open-source data observability for analytics engineers

  1. https://github.com/elementary-data/elementaryelementary

Workflow/Pipeline/MLops

Crontab
Jenkins
Github action
Airflow
mage-ai

The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.

  1. https://github.com/mage-ai/mage-ai
wandb

A tool for visualizing and tracking your machine learning experiments.

  1. https://github.com/wandb/wandb
Drone CI
Flyte
  1. An open-source orchestrator that facilitates building production-grade data and ML pipelines
datachecks

Datachecks is an open-source data monitoring tool that helps to monitor the data quality of databases and data pipelines. It identifies potential issues, including in the databases and data pipelines. It helps to identify the root cause of the data quality issues and helps to improve the data quality.

  1. https://github.com/waterdipai/datachecks

Logs

hydra

A framework for elegantly configuring complex applications

  1. https://github.com/facebookresearch/hydra
whylogs

The open standard for data logging

  1. https://github.com/whylabs/whylogs

#git

Articles

  1. 菜雞新訓記 (1): 使用 Git 來進行版本控制吧
  2. Git Commit Message 這樣寫會更好，替專案引入規範與範例
  3. How to write a good commit message | git tutorial
  4. Conventional Commits 1.0.0
  5. requarks/changelog-action
  6. 🔄 全面掌握 Git 撤銷：詳細解析在 4 種不同層級下的操作方式
  7. Igouist/hello-git

Commit message

  1. pre-commit
  2. Husky
  3. I Do this for every commit and why you should too!
  4. MichaelCurrin/auto-commit-msg
  5. Git Squash: How to Condense Your Commit History
  6. Smoother rebases with auto-squashing Git commits
  7. Git Squash Simplified

Commit message Convention

Type Description
feat Add new features
chore Build process or auxiliary tool changes
fix Fix some bugs
refactor A code change that neither fixes a bug or adds a feature
style Change coding style
docs Write new documentation

Branch Naming Convention

Type Description
wip Works in progress
feat Feature
bug Bug fix or experiment
junk Throwaway branch created to experiment

Rebase

Commands Description
pick 保留該提交，不進行任何更改
reword 修改該提交的提交訊息
edit 挑選該提交，並在應用該提交時停止，以便你可以進行修改
squash 將該提交與前一個提交合併成一個提交
fixup 與 squash 類似，但用於合併提交而無需保留該提交的提交訊息
exec 執行一個 shell 命令。這允許你在 rebase 過程中執行自定義的命令