Inaccurate - Chiappa Firearms X-Caliber 12ga/357/38 Gauge Adapter Insert

I purchased this a couple of weeks ago and have tried to get it to be accurate but have not been able. I have shot over 50 rounds through it but have only been able to shoot to about 3 inch groupings at 25 yards. I thought it might just be my shooting ability with only being able to aim with the bead sight so I took it to a local gun smith about getting better sights on my savage single shot. The gun smith told me it will never achieve much accuracy because the wind disturbance in the barrel after the bullet it leaves the insert. It makes sense but I had not thought of that before. Has anyone else been told this or have "evidence" of this?