I have been thinking about a couple of new numbers that would be nice to have available from the "progress" page. I think we may have discussed these before, but I don't remember who originally suggested them.
The definition of "learned" is necessarily kind of arbitrary, so it would be very useful if we had a statistic which measured the retention rate for "learned" items (like the one provided by Anki's "Deck Statistics"). That is, if over some span of time I reviewed 100 items which I had already "learned", and I got 80 of them correct, my "learned retention rate" for that period would be 80%.
If it's expensive to compute for the entire period, this wouldn't necessarily have to be calculated over all items reviewed in the period, just over some reasonably large random sample; something like 100 items would give a margin of error of less than a percentage point.
Anki also measures the percent correct for what in our case would be "new items" and "non-learned items", i.e. items which we've never seen before and items which we've seen before but are not "learned". These numbers would mostly be useful for comparison with the "learned retention rate" so we can see what "learned" actually means. For example, if my "new items retention rate" is 10%, "non-learned items retention rate" is 30%, and "learned items retention rate" is 80%, I can be fairly confident that "learned" actually means something significant, and it means that I have a roughly 80% chance of getting a prompt right for a "learned" item.
James