Will a mechanical engineer be successfull at doing a arcflash study using a software ? We don't have EE to do the job. Our ME will have a couple of electricians to gather the information and any other help.

Likely not. heck many EE's struggle with it. Contract it out, your guys can still do the data collection but hire a firm that specializes in arc flash studies. And check references, some of the companies doing these are clueless too.

I would not blindly trust the results of a 'self-taught' rookie arc flash analysis.

An arc flash study is not as simple as entering data and pushing a button.
every study requires some amount of educated assumptions to be made. In order to determine if the study results are valid some sensitivity analysis must also be performed.