A token account can also be a program-derived address of other programs than the associated token program. So being off the curve doesn't tell you enough.
For example, a vault program could store tokens in a PDA that isn't an ATA.
You cannot determine from the address alone if it is an ATA, because you cannot reverse the hash process.
However, if you fetch the token account and decode its field you can!
Once decoded you get access to the token account mint and owner, as a result, you can verify that this specific mint and the owner would result in the given token account address.
This is the snippet of the process
import { PublicKey, clusterApiUrl } from "@solana/web3.js"
import { getAssociatedTokenAddressSync, getAccount } from "@solana/spl-token"
const connection = new Connection(clusterApiUrl("mainnet-beta"));
const address = new PublicKey("<TheAddressHere>")
const tokenAccount = await getAccount(connection, address);
const expectedAta = getAssociatedTokenAddressSync(
tokenAccount.mint,
tokenAccount.owner
);
if (expectedAta.equals(address)) {
console.log("It is an ATA");
} else {
console.log("Not an ATA");
}