Muscle cells require more energy to maintain than fat cells, so people with a higher muscle to fat ratio tend to have a higher BMR.
As we get older, we tend to gain fat and lose muscle. This explains why the BMR tends to decrease with age.
In general, men tend to have a faster metabolism as they have more muscle mass, heavier bones and less body fat than women, which is why their daily calorie allowance is higher.
Your metabolism may be partly determined by your genes, although this is not yet fully understood. Genes definitely play a role in muscle size and your ability to grow muscles, both of which affect your metabolism.