I'd like to ask you a few questions to expand on this point, because I never want to be in the OP's situation of feeling like giving up (nowhere near that, luckily)
1. What would you say led to their successes and failures? For example, would you say that the stress of a day job took away their motivation or energy to work on their goals?
2. Do you have any examples of these incidents which caused a change in their approach or perhaps examples of what exactly they changed about their work?
3. Did these acquaintances achieve their success before our current age of Internet/New Media, in other words, before there was a wealth of competition?
4. What advice would you give to someone in the OP's position, or even just someone who fears their amateur work will never progress into professional work?
I could write a book to answer these questions! I'll try and keep my answers brief but in doing so I may end up being a little cryptic or dumbed down. As background, I studied music performance at one of the UK's (and Europe's) top conservatoires and later I was married (and worked with) to one of the top international concert soloists. Combined, these experiences allowed me to meet, socialise and/or work with many of the best amateurs, aspiring pros and most successful pros. World class pros like Rostropovich, Leonard Bernstein and Placido Domingo, through to those like Dave Gilmore, Peter Gabriel, Bjork, Norman Cook (FatBoy Slim) and many others (both luminaries and more "average" pros).
1. There is an almost infinite number of answers to this question. If I were to distil it down to a generalisation, I would say; approach and attitude. The overwhelming majority of amateurs have an amateur approach and attitude. No matter how much natural talent or how skilled one becomes at playing a musical instrument (or filmmaking) the best one can hope to achieve with an amateur attitude and approach is to be a very talented/good amateur musician (filmmaker). The mistake made by so many is that they make incorrect assumptions regarding the difference between "amateur" and "professional". They make these incorrect assumptions based on the appearance of various professional musical performances (or films) but musical performances (or films) are not defined by natural (or acquired) talent! They are defined by a general practise regime and specific preparation, in other words, they are defined by everything the musician has done behind the scenes leading up to the performance. And, it's here, in this behind the scenes preparation, where the biggest difference between amateur and professional exists! There are always areas/aspects of playing an instrument and performing where one is stronger and other areas/aspects where one is weaker and no one, not even the greatest musicians, are naturally gifted enough to not have to work extremely hard. But, even this is not necessarily the main difference because many serious amateurs work extremely hard too. The difference is in how that work is targeted and what it's aims are. In general, amateurs either don't consciously target their work at all or if they do, they tend to dedicate a disproportionate amount of their time/effort/work to those areas which they enjoy the most, rather than to those areas which are weakest and/or require the most time/effort/work! Furthermore, the aims/goals of that work needs to be realistic; ultimately it comes down to risk identification and management, although few musicians (or filmmakers) would put it in those terms. For the professional, this means achieving a certain level of predictability/consistency. To a significant extent, amateur performances are judged by what the performer does well, by the performer's strengths, while professional performers are judged more on what they don't do well, on their weaknesses.
2. Yes, I have quite a few examples but citing one or even a couple of them would give an incomplete or misleading picture and to be honest I don't have the time. All of them though involve some usually minor incident which caused a "light bulb" moment and a change in approach/attitude.
3. I'm not sure this question is entirely relevant because I don't believe there is a greater wealth of competition today than before the internet age. In the 80's and earlier, large numbers of people wanted to be a rock star or Hollywood star (actor or director), probably not much different numbers to today. What has changed is that dramatically more now have the means to actually create some sort of product rather than having to wait to "get noticed" and then being nurtured through the product creation process. This might sound like an improvement but it's very much a two-edged sword!
4. I believe that a large part of the problem is the premise of the question itself, the premise that amateur work can/does "progress to professional work". Switching over to filmmaking specifically, rather than generalisations which can include filmmaking, I don't believe amateur work does or ever has led directly to professional work or at least not for a hundred years or so. Amateur filmmaking and commercial filmmaking are two different things, with different techniques, different working practises, different priorities and different goals. For this reason, why would amateur filmmaking lead to professional filmmaking? Sure, there are a handful of exceptions over the past 25 years or so but one can't logically base a realistic goal on a one in a several million exception!
So, what can the OP do? I don't know much about the OP, so basing it on a generalisation I would suggest the following:
First, be sure you really want to be a professional! It's not uncommon for amateurs who do eventually manage to secure a full-time professional role to then leave the industry after a while because they discover that being a professional is not what they expected from their experience of amateur filmmaking. In other words, many amateurs don't really want to be a professional, what they really want is to be a paid, full-time amateur but within the industry, there's really no such thing as a professional amateur-filmmaker.
Second, if you do want to be a professional then you're going to need to develop a professional attitude and approach. You'll also need an understanding of professional practises/workflows and expectations, so finding an internship or job as a runner to start with is a sensible approach. The OP stated that he has been unsuccessful in securing such a position. There are a number of possible reasons but the most likely is that he isn't selling what the buyers want to buy! Maybe the OP is selling himself as an amateur filmmaker or aspiring pro filmmaker but the "buyers" aren't looking to buy an amateur filmmaker, they are looking to buy a runner! In other words, find out what your "buyers" want to buy and then sell that to them (and nothing else!). In the case of a runner, that probably means you need to be selling: discretion, invisibility, punctuality and reliability, hard work without limitation and nothing else. Although you need to do your research and find out EXACTLY what you need to be selling to your targeted buyers. It's a reasonably safe bet though that selling them an amateur/aspiring filmmaker who's likely to distract them at the wrong time with newbie questions won't get you far!
G