Several parts combine into one character.

第 is a phono-semantic character: 竹 (bamboo) for meaning and 弟 (younger brother) for sound. The 弟 (younger brother) seen earlier originally meant "sequence, order"; to make that ordering sense clear, bamboo (竹) was added to form 第 — hence "order, grade, ordinal." The sense of sequence survives from binding bamboo slips (竹簡) in order.

Korean hanja-sound "je": 第一 (jeil, the first / best), 及第 (geupje, passing an exam), 落第 (nakje, failing), 第三者 (jesamja, a third party). As a prefix it forms ordinals — 第二次 (the second), 第三世界 (the Third World).

Mandarin: dì, 4th tone. As an ordinal prefix it marks nearly every sequence — 第一 (dì-yī, first), 第二 (dì-èr, second), 第三 (dì-sān, third). In modern Chinese it is the most basic character for "the Nth."

Japanese: on-readings ダイ (dai) and テイ (tei) — no kun-reading (on-only). 第一 (daiichi, the first), 次第 (shidai, order / depending on), 及第 (kyūdai, passing), 落第 (rakudai, failing), 第三者 (daisansha, a third party). Japanese 次第 (shidai) is widely used for "depending on / gradually."

Memory aid: bamboo (竹) added to the order-sense of younger-brother (弟) — the 第 of sequence and grade.