First of all, I'd not be sure that there is a reliably standard method for doing the conversion from a text program line to the tokenised version, so each implementation of BASIC could be different.
You don't specify which version of BASIC you're talking about.
Usually, keywords are converted. Literal text, i.e. strings, may be in the nine as literal, but a variable may be actually stored somewhere else. Numeric variable, I'm not sure. How would you count these?
You may need to investigate, and work it out.
Does your BASIC allow the return of the address of a variable, even a $?
You may need to create a line of code with a $ at the start and finish, then get the address of the two marker $, then display the RAM contents from before the first to after the last marker and see just what is there, how your BASIC stores the data, and the full line.
Or does the manual for your BASIC define any of the details already?
Edit: You might find some useful info here, although this relates to GW-BASIC. While the details may be different, many of the general principles may be similar.
http://www.chebucto.ns.ca/~af380/GW-BASIC-tokens.htmlGeoff