Comments on: The Skills Gap For Fortran Looms Large In HPC https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Tue, 27 Jun 2023 23:48:23 +0000 hourly 1 https://wordpress.org/?v=6.5.5 By: Timothy Prickett Morgan https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-210446 Tue, 27 Jun 2023 23:48:23 +0000 https://www.nextplatform.com/?p=142324#comment-210446 In reply to Daniel Williams.

Love this!

]]>
By: Daniel Williams https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-210387 Mon, 26 Jun 2023 16:38:52 +0000 https://www.nextplatform.com/?p=142324#comment-210387 In reply to Thomas Hoberg.

Towards the end of my first decade as a C programmer I was hired into a group writing military simulations in Fortran in the 80s. They were dependent on a huge legacy of Fortran code and the math libraries available. They were porting from VAX to Sun workstations. But their simulation required a specialty UI developed for basic menus and command interface and it did not exist for the Sun systems. They would have to hire the author of that system to come out of retirement and port his code. I looked at the problem as a C programmer and scoffed at the need for this. I told my boss that I would drag the group into the 70s and write an open source X motif wrapper for their menu system. I needed C routines to link the motif code to, and so I started trying to link the Fortran binaries with my C UI routine. I used the C-preprocessor to read the Fortran program’s common block and generate a matching C structure that was an overlay of the internal memory of the Fortran code and allowed the C wrapper to communicate with the Fortran code. I then added a Motif menu system that could read the old definition language they used for their previous menu systems. Tada! Drop in menu replacement using their legacy product and code.
I thought that was cool but my boss was stunned when he figured out I was using code, to generate code on the fly (Compile time). From then on my little kludge was an advertised new feature of their 4th generation code.

]]>
By: Timothy Prickett Morgan https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-209239 Sun, 28 May 2023 02:09:43 +0000 https://www.nextplatform.com/?p=142324#comment-209239 In reply to Steve Suehs.

I, too, have two bows, two swords, and know how to knap flint. Perhaps we don’t trust civilization as much as others? HA! Or we are just retro and curious. Or maybe a little of all of that.

]]>
By: Steve Suehs https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-209223 Sat, 27 May 2023 19:55:39 +0000 https://www.nextplatform.com/?p=142324#comment-209223 I can agree hiring has been painful. I get asked by junior (usually self-taught) devs “should I look into Fortran” which leads to a discussion about there being some money in the Fortran niche but at a cost to your training time, so set your rates accordingly.

I think the tooling story around Fortran is the worst I’ve seen. Tooling has made the Fortran experience stand out as much more unpleasant, even hostile, than most of the other languages I have worked with and I think this contributes to new devs avoiding it once they come into contact with it. The [major vendor] Fortran compiler integration with Visual Studio has been fragile and grueling. If a shop has a big enough Fortran base that they don’t want to just port over, then the code is big enough to warrant tools to navigate, analyze, refactor, debug and test it. None of that analyzes across projects. Yet, it seems there is resistance in the broader Fortran culture and community to advocating for tools, IDE’s, and doc tooling on par with other languages. This has been along the lines of “I don’t need all that popping up in my way.” and “you just need to become a good enough programmer to hold it in your head.” Some of us ask for improved Fortran tools but tool vendors start to see it as niche AND get told the programmers don’t want it, anyway.

I enjoy visiting the renaissance fair, and I know how to knap flint… I had a point here…which is lost now that I realize I am hand-crafting Fortran manipulation tools while I am watching for further reports from my devs that the Fortran debugger slows down and displays incorrect info so I can file a ticket with the vendor. Ah! I don’t want to live in primitive times, knap flint, or write programming language tools, especially for Fortran. Yet, here I am. Vendors are bailing. C++ is a terrible language to parse and analyze and we are awash in tools for C++.

We have met the enemy, and they is us.

]]>
By: Timothy Prickett Morgan https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-208335 Thu, 11 May 2023 14:50:11 +0000 https://www.nextplatform.com/?p=142324#comment-208335 In reply to Thomas Hoberg.

Good heavens, I sure do love reading these kinds of comments. Many thanks to you all who took the time, and continue to do so on other stories. This is a collaborative effort, and I appreciate you all greatly.

]]>
By: Thomas Hoberg https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-208330 Thu, 11 May 2023 13:04:51 +0000 https://www.nextplatform.com/?p=142324#comment-208330 I certainly like the PastPlatforms idea, got plenty to write about, not so sure many would want to read. A bit like ‘Something for the week-end’ on the Reg, I guess.

What I find missing from the discussion is the culture aspect of Fortran. Fortran was the very first attempt to do better than machine or assembly language, which was still a lot better than the true Harvard design with external sequencing logic.

The entire discipline of software engineering hadn’t been invented yet when the first generations of Fortran programmers went to work.

And those guys were physicists, mathematicians, engineers, some of them even linguists (ok, those Chomskys might have been the first to defect to Algol camp): they wanted to solve a problem, they were no Donald Knuth, no Edsger Dysktra, or any of the other Gods who had invented computer science by the time I studied it twenty years later.

(Funnily enough those engineers and scientists seem to have gone from Fortran straight to Python…, sorry I couldn’t help myself)

Looking at the posts here, “Fortran is so much faster” is still a recurring theme, yet since I spent at least some of my many years at University generating compilers, I was inclined to just scoff at that sentiment: by the time the code generator goes to work the original programming language is typically long lost and the GCC or LVM back-ends will generate the same efficiency for any binary, no matter what the original imperative language source code was. There is no inherent advantage to Fortran over C or ADA in terms of generated code (can’t blame array order for everything), we’re not talking Python here.

But then I remembered how we got imbued with a love for recursion, for pointers, trees and all kinds of data structures far beyond arrays, common blocks or even Cobol records in our computer science classes. And I do recall, that I tried to trick that Fortran-77 compiler on the PDP-11/24 into recursion: It would catch any direct attempt, but all it took was an intermediate proxy function to see how quickly the stack on a (16-bit) PDP-11 would bomb.

The elegance or even efficiency of coding in Pascal or Modula, perhaps even Lisp drove our intellectual development, not finishing a science or engineering problem, before our batch job got kicked for running out of the allotted compute slice.

C programmers ran wild with pre-processors, Ratfor seems to have turned everyone into a language designer for some time, with code that’s perhaps deservedly unrecoverable even by the best AI. Some C guys went crazy with bit masking and shifting and outworldish optimizations that were unfortunately impossible to understand a few days later. Some of my dearest colleages spent hours trying to save every bit and every statement, others functionalized every little action, but both never went through the trouble of reading the generated machine code, which sometimes anihilated all their effort, sometimes just inlined or turned a tail recursion into the loop it was meant to be. It’s funny to hear that common sub-expression elimination is a no-no on CUDA cores, because VRAM is so much slower than re-computing in the register file…

But I also remember some C-code that needed to do some bit masking for ISO 8583 messages and used logarithmic functions to get there: I immediately knew it could have only been programmed by a guy with a physics PhD! (he then went on to do a bubble sort on data which came from a group-by SQL-query, so I had him fired as an unmanageable risk).

But with Fortran it must have been rather similar, every science discipline and school probably developed their own style and some of the best known and most widely used libraries where those you never had to look into: did I mention that very few programmers, scientist or engineers were touch typists and liked to tire their wary fingers on comments? There is a reason why most variables in Fortran are single letter and it’s not just because Fortran-60 didn’t allow them longer (or later variants only looked at the first six letters).

Reverse-engineering the algorithmic idea in the original author’s head from the written code might be hard for the author himself, if he (or indeed she) was still alive. And then I’ve met far too many students even in my computer science classes, who believed their testing experiments more than my inductive deliberations on the correctness of their code.

I’ve heard from HPC Fortran code that survives, mostly because the human community around that code is still active, brains can still be picked and pickled into some degree of refactoring. But there is a lot of code out there that needs to be actively retired before it harms someone, very little chance it will tell on its own.

]]>
By: Paul Houston Harkins https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-208284 Wed, 10 May 2023 13:33:04 +0000 https://www.nextplatform.com/?p=142324#comment-208284 Thanks to Timothy Prickett Morgan for this important FORTRAN article and for his many important publications nad contributions, including ITJungle.
I started at IBM as a Systems Engineer in 1962 and we were expected to be able to directly support programmers at many IBM customers in many programming languages, including FORTRAN (on the IBM 1620), COBOL, Assember, PL/1, and RPG, after supporting them in Unit Record (punch card machines like the IBM 402 qand 407), and Autocoder and SPS (Symbolic Programming Language) on the iBM 1401 and 1440 computers.

Mostly all out of the IBM reference manuals after perhaps a one week IBM school, and with essentially no help (no internet) at the customer location.

Today, at age 84, I still write IBM i RPG and IBM i COBOL, and (I guess) IBM i C++ (Fully Free RPG) from home on a $250 a month Cloud computer which is a million times faster, and increibly cheaper and easier, than the original IBM computers of the early 1960s.

TPM is very effectively documenting that history as well as the future in the Next Platform.

]]>
By: HuMo https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-208257 Tue, 09 May 2023 21:26:08 +0000 https://www.nextplatform.com/?p=142324#comment-208257 In reply to Jim Dickerson.

And just today (May 9), IBM released fm.code for this purpose, under the watsonx.ai brand (possibly in partnership with Meta), based on some foundation model (fm), but apparently distinct from LLM. Hopefully a trustworthy very-high-level language approach to programming!

]]>
By: Hubert https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-208182 Sun, 07 May 2023 22:29:09 +0000 https://www.nextplatform.com/?p=142324#comment-208182 In reply to John Fredrickson.

Thanks for this perspective! We were talking about it around the lunch table today, and one French engineer essentially agreed quite unanimously that GPT produced good witchcraft, but the French postal worker, retired teachers, tax officer, and criminologist, expressed nuanced enthusiasm, being positive mostly if new AI tools meant they could work fewer hours per week, and fewer years overall, leveraging the augmented productivity that the tech promises, to enhance their quality of life (without layoffs or salary cuts). For the DOE nuclear bomb Fortran at play here, I guess it would be valuable for the Fed boffins to get student interns piloting GPT code production over the summer (with various levels of complexity), test it out against their existing codebases, and report back on the level of success of the enterprise. A win-win as students (the Nation’s future) would get simultaneously trained in GPT/AI, Fortran, and nuclear bombs!

]]>
By: Jim Dickerson https://www.nextplatform.com/2023/05/02/the-skills-gap-for-fortran-looms-large-in-hpc/#comment-208174 Sun, 07 May 2023 17:10:45 +0000 https://www.nextplatform.com/?p=142324#comment-208174 In reply to John Fredrickson.

Thanks John, but I’m talking about breaking the cycle of running out of programmers for older languages. Otherwise, we’ll still be talking about this in 20 years. I think Ondřej Certik is moving in the wrong direction though it may keep him in his comfort zone to stick with what he knows. There are many methodologies for transitioning from legacy code and you have to start sometime. AI has just made it easier.

]]>