I've been using the Acoustic Guitar String Size Calculator at wahiduddin.net to calculate string sizes for various scale lengths and tensions.  You enter your scale, tension, tuning and octave and it will calculate the theoretical string size.  You will have to try different tension values to find your results.  For a 24" scale G2D3G3, low "G" tuning I like to have approximately 21#, 21# & 15# tension respectively to give .044", .028" & .017" string diameters.  This gives 57# total tension on the neck and has worked fine for my 3/4"x1.5" red oak necks.  You may prefer other tensions.  Follow the instructions on the site.

My final decision about string size is based on the strings performance.  If it has undulating undertones that I don't want, I use a bigger string at higher tension for the same note.  I test this by tightening the string until I like its performance, determining the new note it is tuned to and then, work backward on the string calculator to a find the tension.  I use the new tension to find the correct string size for my desired tuning and string performance.

I mostly do finger picking.  Another personal influence is that my thumb and first finger can handle the higher tensions, by middle finger cannot.  So, for the little string, I usually use the smallest string that will give me acceptable performance.  This is reflected in my choice of strings above.

This Calculator is a good place to start and will keep you from breaking strings or unnecessarily damaging your guitar.  By the way, you can download the Calculator page to your computer and use it without a web connection.

The attached photo is of a 20.25" scale guitar tuned to G2D3G3, low "G" tuning, with .050", .032" & .021" strings.  I can crank the little string up to a G2D3B3 tuning but, it's very tight for my middle finger picking.  As a result of the bigger strings and shorter scale length the 20.25" guitar is more mellow than a 24" guitar.

Uncle Fred