No, your readership is not down. Rather, it has gone up, probably by a lot. You just don't know it.
Many bloggers, including me, use the free Sitemeter widget for tracking website visitors. This is a useful little tool, and has many interesting statistics, even if there is the tendency to obsess over the numbers.
However, many blog readers, including me, and especially those who read plenty of blogs daily, use RSS feedreader software to read their favorite blogs. RSS technology bypasses your attractive blog design and merely posts your articles. Crucially, Sitemeter doesn't track RSS feed readers, and so may miss some of the voracious readers out there. You may be missing the bulk of your readership.
Keep up the good work! Especially Catholic writers, you might be inspired by the Decree on the Media of Social Communications from the Second Vatican Council, Inter Mirifica.