Which image did you use ? firmware.A20-OLinuXino-Lime.img.gz ?
I could install Debian using the A20 OLinuXino-Lime2 image.
Are you sure it is a A20-OLinuXino-Lime and not a A20-OLinuXino-Lime2 ?
Early A20-OLinuXino-Lime2 boards are marked as "A20-OLinuXino-Lime", not "Lime2". A20-OLinuXino-Lime has one DRAM chip (U1) and a RTL8201CL
10/100 PHY (U2) on the top side whereas A20-OLinuXino-Lime2 has two DRAM
chips (U2, U3) on the top side and a RTL8211CL/E or KSZ9031 10/100/1000
PHY (U5) on the bottom side.