See also
Recommended alpha and beta metrics
Comparing alpha diversity between groups
Statistical significance of diversity differences
Alpha diversity
Beta diversity
Abundance bias
Cross-talk
Background
Diversity metrics are calculated from an OTU table containing counts or frequencies. Naively, these counts or frequencies are assumed to be reasonable approximations to the observed numbers of cells or frequencies of species in the sample. This naive assumption is badly wrong due to issues with PCR and sequencing. In fact, the abundance of a species in the OTU table correlates poorly with abundance in the sample due to abundance bias, which is caused by varying copy count (the number of 16S genes ranges from one to ~10 per genome) and mismatches with PCR primers. With fungal ITS, the problem is more severe because the copy count ranges from 10s to 100s of copies of the ITS region per genome, but this issue is not well-understood due to a lack of complete fungal genome sequences.
Spurious counts
Often, counts in the OTU table are entirely spurious and should be zero. This happens for three main reasons: contaminants, spurious OTUs, and cross-talk. Contaminants are self-explanatory.
Spurious OTUs
Spurious OTUs arise from unfiltered errors due to PCR and sequencing. If the recommended OTU clustering pipeline is followed, I believe from multiple lines of evidence that the number of spurious OTUs will usually be small, but this is impossible to verify with certainty for samples collected in vivo. With low-biomass and low-diversity samples, a substantial fraction of OTUs may be spurious.
Cross-talk
Cross-talk occurs when a read is assigned to the wrong sample, e.g. because of a base call error in the index (sometimes also called the barcode or tag sequence). Typically, ~0.1% of reads are assigned to the wrong sample. With sufficiently many reads, cross-talk will cause all OTUs to appear in all samples and the composition of the reads assigned to a given sample converges on the "super-sample" obtained by pooling all samples together. In typical data, many small counts appear to be due to cross-talk, but as with spurious OTU sequences, cross-talk cannot be reliably distinguished from a valid low-abundance signal except in control samples.
Frequency-based diversity metrics
Many diversity metrics including Shannon, Jost, Simpson and weighted UniFrac, are calculated from OTU frequencies. If such a metric is calculated from an OTU table, it is likely be quite different from the true value for the sample because of abundance bias and spurious counts.
Presence/absence diversity metrics
Other metrics, including richness and unweighted UniFrac, are based on presence/absence of an OTUs in the sample. If such a metric is calculated from an OTU table, it is likely to be quite different from the true value in the sample due to three main effects: incomplete sampling, spurious OTUs, and cross-talk. In typical data, many OTUs are probably present in the sample but absent from the OTU table because there are insufficient reads (or amplicons) to account for all of the low-abundance species. Other OTUs are absent because they are not amplified by PCR due to primer mismatches. These effects reduce the number of OTUs in the table compared to true value in the sample. Spurious counts due to cross-talk and spurious OTUs increase the number of OTUs compared to the true value in the sample. The effects of spurious counts can be severe, possibly accounting for half or more of the non-zero counts of a low-diversity sample, even when stringent quality control procedures are followed. The values of metrics based on presence/absence are therefore unreliable, and the uncertainties are not known so error bars cannot be determined.
Diversity metrics cannot be meaningfully measured by amplicon sequencing
The above considerations show that it is impossible to measure meaningful values for any diversity metric using NGS amplicon sequencing. However, it is still possible to compare alpha diversity between groups.