It turns out that it's an issue with the color channels being in sRGB color space and the alpha channel being linear. I verified this by doing a software blend of the images and then doing another software blend using gamma corrected values. There's a setting in opengl to correct for it, glEnable(GL_FRAMEBUFFER_SRGB), but I haven't tried this yet.