Mouse Soup and Mouse Tales, by Arnold Lobel. As far as I can tell, the two books are not exactly related, but they look like a series, are titled like a series, and are meant for the same age group. And both are about mice.
I picked these books up after my son so thoroughly enjoyed the Frog and Toad series by Lobel.
Mouse Soup is clever, although a little strange (by modern standards) at times. I found it to be enjoyable and well-written. It is appropriate for beginning readers, full of short words, repetition, and interesting vignettes.
Mouse Tales was my favorite of the two. It is charming, charming, charming. My son and I read it through multiple times in a row, on our first time. I got a kick out of two of the tales, especially.
I would recommend both books, but especially Mouse Tales, and especially if you have a quirky sense of humor and an old-fashioned sense of charm.