pub fn wordpiece_tokenize(word: &str, vocab: &Vocab) -> Vec<String>Expand description
Tokenize a single word into WordPiece tokens.
Returns a vector of subword token strings such as ["un", "##aff", "##able"].
If the word cannot be tokenized, returns ["[UNK]"].