Using annotate to place a text box below legend in

I wish to display some text in a Matplotlib plot using annotate(), aligned beneath a legend box. I have examined the solution proposed in How to place a text box directly below legend in matplotlib? and can make an adaption that works for my situation, however adding additional axes etc. seems overkill for my situation. I simply wish to place an annotation whose top right corner is 20 pixels below the lower right corner of the legend box. I also wish to save a PDF files of the plot.

Here is my code.

import matplotlib.pyplot as plt
import matplotlib.text

fig, ax = plt.subplots()

ax.plot([5,1], label="Label 1")
ax.plot([3,0], label="Label 2")

legend = ax.legend(loc="upper right")

# Create offset from lower right corner of legend box,
# (1.0,0) is the coordinates of the offset point in the legend coordinate system
offset = matplotlib.text.OffsetFrom(legend, (1.0, 0.0))
# Create annotation. Top right corner located -20 pixels below the offset point 
# (lower right corner of legend).
ax.annotate("info_string", xy=(0,0),size=14,
            xycoords='figure fraction', xytext=(0,-20), textcoords=offset, 
            horizontalalignment='right', verticalalignment='top')
# Draw the canvas for offset to take effect
fig.canvas.draw()

fig.savefig('plot1.png')
fig.savefig('plot1.pdf')

plt.show()

When the plot is first shown it looks correct, with the information text correctly beneath the legend. Also, the saved png file shows correctly. However, the text is missing in the pdf output.

I am not sure if there are gaps in my understanding, or if something is wrong. Can anyone shed some light why the pdf output is incorrect?

png output