The problem is probably either a saw that is too wide or wood that is too soft. I have rescued the odd misscut fret by back filling with superglue and sanding dust and recutting.
I use a standard dovetail saw but have used hack saw blades. If the steel on your saw is thin enough you might narrow the kerf it cuts by hammering it slightly flat. I have a special tool called a saw set to spread the teeth sideways when I have to sharpen the saw and the kerf becomes too tight for the fret wire.
You can try gluing in the frets. I always use a fretboard. An oak fretboard would also add a lot of rigidity to your neck. Over time the poplar will bow just a bit...
I really like a maple neck.. super hard, smooth...
a spot of super glue will work. leveling the frets makes or breaks a fretted instrument. best thing i ever brought was a fret file. good luck and don't be put off, its fiddly, and adds hours to the build time but it is worth it.
Poplar is WAY too soft for fretboards - so soft that the slot won't really grip the tang of the fretwire and it is very easy to hammer or press the frets into the surface of the board and end up with uneven frets. It is also essential that your saw is cutting exactly the right width of slot - just a few thou of an inch too wide is enough to cause a problem. Here's how I install frets:
I have successfully fretted poplar.
Are you using a miter box or some other means to keep the saw stable and not letting it drift out of square? If it drifts from square to the neck you get an hourglass slot that's wider at the edges. If it drifts from square to the fingerboard the slot will be V shaped instead of U shaped and wider at the top where the tang barbs are supposed to grab the wood. and finally are you making sure the slot is deep enough for the whole tang to go all the way down?
Yes, I have a miter box the make sure everything is in line.
Thanks for the tips from everyone, looks like I'll have to scrap it and start with a harder wood, probably oak. I used a scrap piece of oak with the same setup to do a couple of test runs and in seemed to take care of the problem. So here i go again, I guess.