Transaction validation is the process of determining if a transaction conforms to specific rules to deem it as valid. Validators check if transactions meet protocol requirements before adding the transactions to the distributed ledger as part of the validating process.
This validation process is carried out by nodes who store full copies of the blockchain. When nodes validate a transaction, it is added to the mempool (short for memory pool). In a proof of work network, miners are incentivized by transaction fees to confirm these transactions by including them in a block in the blockchain, establishing a clear chronological record of when the transaction occurred so that a later transaction cannot spend the same coins as in the original.
A transaction is considered valid if the sender in the transaction has an initial balance in their wallet equal to or greater than the amount being sent in the transaction (including the transaction fee). Other rules can exist depending on the specific protocol in question, but this rule is generally applicable to all protocols.
An example of a scenario that includes a transaction which should be deemed invalid by a cryptocurrency protocol is the so called double-spend attack. The first transaction in this scenario is a valid transaction in which coins are spent from an address that has an adequate balance to fund the transaction amount. However, the malicious actor in this case then tries to prevent this valid transaction from being included in the distributed ledger. They do this by controlling a majority of the mining power in the network, thus allowing them to mine the longest version of the blockchain (i.e. the valid chain according to Nakamoto Consensus). If successful, the malicious actor will then be able to spend the same coins from the initial transaction a second time in what would be an invalid transaction if the original valid transaction had been included in the ledger.