Thank you Alberto. Your post was key to my understanding what was going on.
Actually someone suggested __attribute__((packed)) yesterday and it fixed the problem (I was planning on updating the post today anyway), but I didn't understand why there was a discrepancy between what was in memory and what the compiler thought was in memory.
I looked at how the data was getting put into memory, and then the cause was clear. To put the data in memory, I am reading fields from a database one by one, incrementing a pointer to memory by the size of the data type of the field each time. So when I put the data in memory there is no 4 byte gap before the double because I didn't create one.
Rather than trying to guess where the compiler is going to insert gaps and create them when I put the data into memory, it seems like the better solution is to use __attribute__((packed)).