ejolson
Posts: 3246
Joined: Tue Mar 18, 2014 11:47 am

Re: 64-bit operating system

Tue Mar 20, 2018 6:46 am

jdb wrote:
Mon Mar 19, 2018 10:45 pm
Ok, so "database engines that memory map inordinately huge database files" is one valid reason. Any more?
The advantage of 64-bit mode is not only 64-bit memory addresses, but also 64-bit integers in general. Any compute-bound program that makes heavy use of 64-bit integers will show significantly better performance in 64-bit mode.

A bad but common example of a 64-bit integer computation is the prime number finder that sysbench uses to determine CPU speed. Sysbench runs about 15 times slower in 32-bit mode compared to 64-bit mode. While the sysbench CPU score doesn't mean very much, this fact is not obvious to many technology bloggers. The result is that the Raspberry Pi is frequently described as being much slower than other single-board computers when, in fact, its speed is almost exactly the same.

At the same time, it seems strange to default to a 64-bit operating system because the technical understanding of technology bloggers is so low.

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: 64-bit operating system

Tue Mar 20, 2018 8:12 am

jdb wrote:
Mon Mar 19, 2018 9:22 pm
In what way is the current ARMv7 Raspbian bottlenecked or limited compared to a theoretical ARMv8 Raspbian?
I see there is kernel7.img, but the user space programs are all ARMv6 are they not? so they can run on the Pi Zero.
ejolson wrote:
Tue Mar 20, 2018 6:46 am
The advantage of 64-bit mode is not only 64-bit memory addresses, but also 64-bit integers in general. Any compute-bound program that makes heavy use of 64-bit integers will show significantly better performance in 64-bit mode.
Its more than that, much more.

1) There are double the number of registers, both integer and floating point. That is 31 64-bit integer registers plus the zero register, compared to 16 32-bit integer registers. (The zero register on 64-bit is very useful indeed).
One example: if you think about how many scalar variables and arguments you typically have in your functions, you will see that the number of cases where the stack needs to be adjusted is far less in 64-bit mode.

2) The modern IEEE compliant version of NEON is used in 64-bit mode. The SIMD registers now have the obvious sensible layout. The presence of NEON is now guaranteed, it is no longer a co-processor.

3) The instruction set has been streamlined and modernized, and in particular is now suitable for the higher performance arm processors.

4) The assembly language has been cleaned up, the historical cruft removed, and is generally easier to read and use.

To be fair 64-bit integers work fine in 32-bit mode and are very usable, they are just slow.

YMMV but programs seem smaller, presumably because of the more powerful ISA, or larger register file.
One program I have in front of me is 78k for 32-bits and 72k for 64 bits (but it does do a little 64-bit arithmetic).

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2035
Joined: Thu Jul 11, 2013 2:37 pm

Re: 64-bit operating system

Tue Mar 20, 2018 8:56 am

ejolson wrote:
Tue Mar 20, 2018 6:46 am
jdb wrote:
Mon Mar 19, 2018 10:45 pm
Ok, so "database engines that memory map inordinately huge database files" is one valid reason. Any more?
The advantage of 64-bit mode is not only 64-bit memory addresses, but also 64-bit integers in general. Any compute-bound program that makes heavy use of 64-bit integers will show significantly better performance in 64-bit mode.

A bad but common example of a 64-bit integer computation is the prime number finder that sysbench uses to determine CPU speed. Sysbench runs about 15 times slower in 32-bit mode compared to 64-bit mode. While the sysbench CPU score doesn't mean very much, this fact is not obvious to many technology bloggers. The result is that the Raspberry Pi is frequently described as being much slower than other single-board computers when, in fact, its speed is almost exactly the same.

At the same time, it seems strange to default to a 64-bit operating system because the technical understanding of technology bloggers is so low.
jahboater wrote:
1) There are double the number of registers, both integer and floating point. That is 31 64-bit integer registers plus the zero register, compared to 16 32-bit integer registers. (The zero register on 64-bit is very useful indeed).
One example: if you think about how many scalar variables and arguments you typically have in your functions, you will see that the number of cases where the stack needs to be adjusted is far less in 64-bit mode.

2) The modern IEEE compliant version of NEON is used in 64-bit mode. The SIMD registers now have the obvious sensible layout. The presence of NEON is now guaranteed, it is no longer a co-processor.

3) The instruction set has been streamlined and modernized, and in particular is now suitable for the higher performance arm processors.

4) The assembly language has been cleaned up, the historical cruft removed, and is generally easier to read and use.

To be fair 64-bit integers work fine in 32-bit mode and are very usable, they are just slow.

YMMV but programs seem smaller, presumably because of the more powerful ISA, or larger register file.
One program I have in front of me is 78k for 32-bits and 72k for 64 bits (but it does do a little 64-bit arithmetic).
Ok, so the instruction set is "nicer" and sysbench is inherently faster. Does the "niceness" of the ISA translate into any of the typical workloads of a Pi (Kodi, Chrome browser, JS JIT, Python interpretation)?

Are there any other real-world benchmarks that show significant differences between ARMv7 NEON and ARMv8 NEON? If so, post some numbers.
Rockets are loud.
https://astro-pi.org

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: 64-bit operating system

Tue Mar 20, 2018 9:10 am

jdb wrote:
Tue Mar 20, 2018 8:56 am
Ok, so the instruction set is "nicer" and sysbench is inherently faster. Does the "niceness" of the ISA translate into any of the typical workloads of a Pi (Kodi, Chrome browser, JS JIT, Python interpretation)?
Surely 64-bit benefits (such as double the number of registers) would apply equally to all code running on the machine? There would be no legacy code of course. We are talking very very approximately about only 15% or so (which is about what you get with the new 3B+ :) )

Sysbench does a lot of divisions and there is no division instruction on ARMv6 which it is compiled for.
jdb wrote:
Tue Mar 20, 2018 8:56 am
Are there any other real-world benchmarks that show significant differences between ARMv7 NEON and ARMv8 NEON?
Good question, I doubt it.
It is not ARMv8 NEON in general, it is ARMv8 NEON in 64-bit mode where they fixed things like the register layout (presumably so there would be no backwards compatibility issues). There are a few nice new instructions in ARMv8 neon available in both modes but I don't think they would make much difference.

I like Aarch64 because it provides a "baseline", you can assume a decent, modern, and complete instruction set.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23082
Joined: Sat Jul 30, 2011 7:41 pm

Re: 64-bit operating system

Tue Mar 20, 2018 9:39 am

It's possible it comes down to a simple question.

If you REALLY need the extra speed that 64bit processing gives you, why are you not using a faster machine? I'm not talking a faster SBC, but a proper full on high spec device ie desktop or laptop with an i7 or i9? Because if that speed is so important, it should be important enough to get the right machine for the job.

It's a bit like the continual questions demanding why we don't have more RAM on the pi (ignoring the HW limitations which people seem to forget). If you have a use case that demands that extra RAM, then get a machine with the required amount of RAM.

Or just write more efficient code (either memory or speed).

After all, we can now decode 1080p30 10bit HEVC video on the Pi. That is a HUGELY complicated computational task. More so that almost ANYTHING people on here would ever approach. How do we do it? Optimised code using NEON, along with judicious use of HW where possible.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: 64-bit operating system

Tue Mar 20, 2018 9:42 am

jamesh wrote:
Tue Mar 20, 2018 9:39 am
If you REALLY need the extra speed that 64bit processing gives you, why are you not using a faster machine? I'm not talking a faster SBC, but a proper full on high spec device ie desktop or laptop with an i7 or i9? Because if that speed is so important, it should be important enough to get the right machine for the job.
Please don't bring common sense into it :)

We love the Pi (and Raspbian).

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23082
Joined: Sat Jul 30, 2011 7:41 pm

Re: 64-bit operating system

Tue Mar 20, 2018 10:07 am

jahboater wrote:
Tue Mar 20, 2018 9:42 am
jamesh wrote:
Tue Mar 20, 2018 9:39 am
If you REALLY need the extra speed that 64bit processing gives you, why are you not using a faster machine? I'm not talking a faster SBC, but a proper full on high spec device ie desktop or laptop with an i7 or i9? Because if that speed is so important, it should be important enough to get the right machine for the job.
Please don't bring common sense into it :)

We love the Pi (and Raspbian).
Don't get me wrong - the Pi is great, and works in a huge number use cases - more than most people who claim they need 64bit realise. And keeps me in beer and skittles.

But horses for courses.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
pi-anazazi
Posts: 518
Joined: Fri Feb 13, 2015 9:22 pm
Location: EU

Re: 64-bit operating system

Tue Mar 20, 2018 10:08 am

Ok, I have here a Raspi 3 with 64 bit

Code: Select all

Linux rasp3TW0218 4.15.7-1-default #1 SMP Wed Feb 28 12:40:23 UTC 2018 (a36e160) aarch64 aarch64 aarch64 GNU/Linux 
In everyday use I don't see any "fast-as-hell" compared to standard Raspbian stretch.

Would anybody give me a short hint to a "speed test" (some "make this or that"?)? To get some numbers for this discussion? :-)
Kind regards

anazazi

User avatar
piglet
Posts: 900
Joined: Sat Aug 27, 2011 1:16 pm

Re: 64-bit operating system

Tue Mar 20, 2018 10:14 am

W. H. Heydt wrote:
Mon Mar 19, 2018 4:27 pm
I once worked on an IBM mainframe (S/360-30) that had 32KB. It really hurt when a new release caused the OS nucleus to grow from 6KB to 8KB. I spent several weeks recompiling programs to get rid of double-buffering of files so they would still fit and run (no virtual memory in those days...and, as an aside, almost no microcomputers, either).
It happened again when we had to start to worry about "above the line" and "below the line" on those massive Mainframes that moved beyond 16MB. Oh how spoiled we were with 32MB memory to play with! RMODE(31)

1GB of memory? That was just someone smoking something in their tobacco at the terminal beside you. Yes. Back then we had to work in closed rooms with people smoking cigarettes and pipes all day every day...and it was seen as normal.

Heater
Posts: 12972
Joined: Tue Jul 17, 2012 3:02 pm

Re: 64-bit operating system

Tue Mar 20, 2018 10:19 am

It's easy,

Mo' bits, mo' better!

Anyway, what's the problem? Pi works fine at being what it is, as it is. If you really want a 64 bit OS on it for some odd purpose then they are available.
Last edited by Heater on Tue Mar 20, 2018 10:39 am, edited 1 time in total.

Heater
Posts: 12972
Joined: Tue Jul 17, 2012 3:02 pm

Re: 64-bit operating system

Tue Mar 20, 2018 10:25 am

Ha, piglet, I recall a woman proudly telling me her PC had 40MBytes of RAM. Circa 1995. I was impressed as my humble machine had far less.

Turned out she did some kind of IT management in some company and had pulled all the "unused" RAM from the office workers machines and installed it in her machine.

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: 64-bit operating system

Tue Mar 20, 2018 10:41 am

jamesh wrote:
Tue Mar 20, 2018 10:07 am
Don't get me wrong - the Pi is great, and works in a huge number use cases - more than most people who claim they need 64bit realise. And keeps me in beer and skittles.
Your right, just get a more powerful machine if you really need the speed.

But compared to a powerful x86 machine the Pi is physically tiny, low power consumption, and very very cheap.
Which is attractive in some cases.

You mentioned in the past how difficult to program NEON is, which is true.
But have you looked in detail at SSE4.2 say and now AVX512 ? it is absolutely truly comprehensively horrible compared to NEON.

Anyone wanting to learn SIMD programming would do well to start with NEON.

Sure AVX is vastly more capable and powerful than NEON, but give me NEON any day, especially in 64-bit mode where the syntax is easier.

Wandered off topic, sorry.

mfa298
Posts: 1387
Joined: Tue Apr 22, 2014 11:18 am

Re: 64-bit operating system

Tue Mar 20, 2018 12:55 pm

jdb wrote:
Tue Mar 20, 2018 8:56 am
Are there any other real-world benchmarks that show significant differences between ARMv7 NEON and ARMv8 NEON? If so, post some numbers.
I have seen some comments that for a particular Crypto Currency with a Proof of Work algorithm tailored to being used on lower powered devices (such as the Raspberry Pi and other SBCs) that a 64bit OS makes a huge difference. I don't have numbers and it's not something I've tried out myself (maybe one of those days when the other projects are done ... )
jamesh wrote:
Tue Mar 20, 2018 9:39 am
After all, we can now decode 1080p30 10bit HEVC video on the Pi. That is a HUGELY complicated computational task. More so that almost ANYTHING people on here would ever approach. How do we do it? Optimised code using NEON, along with judicious use of HW where possible.
The question that might raise is whether it could work even better on a 64bit OS (assuming there aren't other issues).

Overall I think there's a few questions (and likely answers) here:
  • Do the majority of users need a 64bit OS - Probably not
  • Is there a potential for a speed up in real world tasks with a 64bit OS - Quite possibly (although how do you actually test that)
  • What's the support overhead - Massive
  • How many people actually care - Not that many

Personally I've not tried any 64bit OS on my Pi's (although I might try, along with netbooting, on the 3B+ I've just ordered). For the most part I'd agree that for the majority of users a 32bit image that's compatible between all boards is a better approach than having an optimised image for each specific version (which would become a support nightmare) but that doesn't mean there's also potential for better performance out of a 64bit image (which will likely also be better optimised for the hardware).

What might be particularly good is some decent guides on how to build various OS options on the Pi - Most of the people who really seem to know how to do it have the information wrapped up in custom build scripts rather than being documented in an easy to digest form.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5784
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: 64-bit operating system

Tue Mar 20, 2018 2:55 pm

Out of curiosity, where is this 15% number coming from?

From the benchmarks I've seen some operations were a little faster, some were a little slower, but there was nothing noticeable like 15% better overall performance or noticeable improvements in everyday tasks. Wish I could find that thread.

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: 64-bit operating system

Tue Mar 20, 2018 3:12 pm

ShiftPlusOne wrote:
Tue Mar 20, 2018 2:55 pm
Out of curiosity, where is this 15% number coming from?

From the benchmarks I've seen some operations were a little faster, some were a little slower, but there was nothing noticeable like 15% better overall performance or noticeable improvements in everyday tasks. Wish I could find that thread.
I have seen it mentioned in these forums somewhere, but here are a couple from google:

https://www.cnx-software.com/2016/03/01 ... tructions/

https://www.cnx-software.com/2015/04/09 ... bit-cores/

Presumably the difference is much greater on the high performance ARM CPU's such as the A57 or the A73.
I guess things like ldm/stm, the ability to conditionally execute almost any instruction, the ability to write directly to the PC, are really bad news for modern out-of-order CPUs, and so they have been removed from A64.

ShiftPlusOne
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5784
Joined: Fri Jul 29, 2011 5:36 pm
Location: The unfashionable end of the western spiral arm of the Galaxy

Re: 64-bit operating system

Tue Mar 20, 2018 3:24 pm

jahboater wrote:
Tue Mar 20, 2018 3:12 pm
ShiftPlusOne wrote:
Tue Mar 20, 2018 2:55 pm
Out of curiosity, where is this 15% number coming from?

From the benchmarks I've seen some operations were a little faster, some were a little slower, but there was nothing noticeable like 15% better overall performance or noticeable improvements in everyday tasks. Wish I could find that thread.
I have seen it mentioned in these forums somewhere, but here are a couple from google:

https://www.cnx-software.com/2016/03/01 ... tructions/

https://www.cnx-software.com/2015/04/09 ... bit-cores/

I expect the difference is much greater on the more recent high performance ARM CPU's such as the A57 or the A73.
Yes, integer performance would go up, so benchmarks like drystone would show improved performance.
viewtopic.php?f=72&t=137963&start=75#p945346

I've email that first cnx-soft article around when it came out, but lost the email explaining why those results are unlikely to translate to real-world performance on the pi.

It may be worth revisiting this and actually making some measurements and running some benchmarks.

ejolson
Posts: 3246
Joined: Tue Mar 18, 2014 11:47 am

Re: 64-bit operating system

Tue Mar 20, 2018 4:53 pm

ShiftPlusOne wrote:
Tue Mar 20, 2018 2:55 pm
Out of curiosity, where is this 15% number coming from?
I don't know about 15 percent, but sysbench on a Pi takes about 126 seconds in 32-bit mode compared to 8 seconds in 64-bit, which is where I got a factor of 15 times.

Another observation is the fact that almost all x86 distributions are becoming 64-bit only. The Linux kernel runs fine on 32-bit. However, it is becoming more and more difficult to compile userland for 32-bit. I suspect because much of userland is developed on machines running in 64-bit mode by people who have a fairly narrow view of hardware.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23082
Joined: Sat Jul 30, 2011 7:41 pm

Re: 64-bit operating system

Tue Mar 20, 2018 5:14 pm

ejolson wrote:
Tue Mar 20, 2018 4:53 pm
ShiftPlusOne wrote:
Tue Mar 20, 2018 2:55 pm
Out of curiosity, where is this 15% number coming from?
I don't know about 15 percent, but sysbench on a Pi takes about 126 seconds in 32-bit mode compared to 8 seconds in 64-bit, which is where I got a factor of 15 times.

Another observation is the fact that almost all x86 distributions are becoming 64-bit only. The Linux kernel runs fine on 32-bit. However, it is becoming more and more difficult to compile userland for 32-bit. I suspect because much of userland is developed on machines running in 64-bit mode by people who have a fairly narrow view of hardware.
Which, considering there are probably more Linux kernels running in 32bit than 64bit worldwide, is a particular odd view to take. Although that depends on how many Android kernels are 64bit I suppose.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

tkaiser
Posts: 103
Joined: Fri Aug 05, 2016 1:28 pm

Re: 64-bit operating system

Sun Mar 25, 2018 4:57 pm

jahboater wrote:
Mon Mar 19, 2018 6:42 pm
There is something odd about ARM addressing, or its memory mapped IO, that requires a 64-bit OS when moving to 2GB
Still not true as already explained: viewtopic.php?f=91&t=192321#p1281326

Those other 64-bit ARM thingies usually combine a 64-bit kernel with a 32-bit (Android) userland. For the simple reason of 64-bit code eating up more memory and no 64-bit benefits with this use case (Android) anyway.

That the OS images for ODROID-C2 and Pine64 are 64-bit ones is just due to the excitement of their respective target audience. Users think 64-bit would be better than 32-bit since twice as much. As can be seen in my referenced post above depending on the use case running a 64-bit userland can result in way more memory needed which is a problem on platforms like RPi which lack decent amounts of memory.

Anyway: Raspbian can't go 64-bit as one of the Raspbian guys explained: https://www.cnx-software.com/2016/03/01 ... ent-523286 (while multiarch works pretty well on upstream Debian/Ubuntu this isn't an option with Raspbian unless support for ARMv6 will be dropped which will of course never happen).

I had my Pi 3 with a 64-bit kernel and an armhf Ubuntu userland running for some time, adding arm64 architecture plus packages was no problem at all. But usually on 64-bit systems it's done the other way around (shipping an 64-bit userland and then use multiarch to add support for older ARM stuff, e.g. the 'famous Plex hack')
Last edited by tkaiser on Sun Mar 25, 2018 5:43 pm, edited 1 time in total.

tkaiser
Posts: 103
Joined: Fri Aug 05, 2016 1:28 pm

Re: 64-bit operating system

Sun Mar 25, 2018 5:00 pm

ejolson wrote:
Tue Mar 20, 2018 4:53 pm
ShiftPlusOne wrote:
Tue Mar 20, 2018 2:55 pm
Out of curiosity, where is this 15% number coming from?
I don't know about 15 percent, but sysbench on a Pi takes about 126 seconds in 32-bit mode compared to 8 seconds in 64-bit
Don't you think that is the result of ARMv8 vs. ARMv6 and not 64-bit vs. 32-bit in this case?

tkaiser
Posts: 103
Joined: Fri Aug 05, 2016 1:28 pm

Re: 64-bit operating system

Sun Mar 25, 2018 5:13 pm

ejolson wrote:
Tue Mar 20, 2018 6:46 am
Any compute-bound program that makes heavy use of 64-bit integers will show significantly better performance in 64-bit mode.
Not all, see the C-Ray test results and explanation: https://libre.computer/2018/03/21/raspb ... omparison/

3 different Cortex-A53 SoCs, all clocked between 1.4GHz and 1.5GHz so the 32-bit vs. 64-bit difference can be studied in detail.

jahboater
Posts: 4595
Joined: Wed Feb 04, 2015 6:38 pm

Re: 64-bit operating system

Sun Mar 25, 2018 5:13 pm

tkaiser wrote:
Sun Mar 25, 2018 5:00 pm
ejolson wrote:
Tue Mar 20, 2018 4:53 pm
ShiftPlusOne wrote:
Tue Mar 20, 2018 2:55 pm
Out of curiosity, where is this 15% number coming from?
I don't know about 15 percent, but sysbench on a Pi takes about 126 seconds in 32-bit mode compared to 8 seconds in 64-bit
Don't you think that is the result of ARMv8 vs. ARMv6 and not 64-bit vs. 32-bit in this case?
Its a bit of both.

Code: Select all

The cpu is one of the most simple benchmarks in SysBench. In this mode each request
consists in calculation of prime numbers up to a value specified by the
--cpu-max-primes option. All calculations are performed using 64-bit integers.
As above, it uses 64-bit arithmetic which is bound to be slower in 32-bit mode.
And the 32-bit version is compiled for ARMv6 that has no integer division instruction, which in a simple prime number benchmark is probably a major handicap. I doubt it uses the sieve method.

It would be interesting to see the result of sysbench compiled for 32-bit ARMv8, which most Pi's run now.

tkaiser
Posts: 103
Joined: Fri Aug 05, 2016 1:28 pm

Re: 64-bit operating system

Sun Mar 25, 2018 5:25 pm

jahboater wrote:
Sun Mar 25, 2018 5:13 pm
It would be interesting to see the result of sysbench compiled for 32-bit ARMv8, which most Pi's run now.
Rather useless: https://forum.armbian.com/topic/1748-sb ... ns/?page=2

It's a compiler and not a CPU benchmark so therefore Raspbian ARMv6 binary performs better than Debian armhf.

timrowledge
Posts: 1272
Joined: Mon Oct 29, 2012 8:12 pm
Location: Vancouver Island
Contact: Website

Re: 64-bit operating system

Sun Mar 25, 2018 7:11 pm

Not sure why I keep seeing assertions that 64bit code is bigger. AArch64 is still 32bit wide instructions. You can still choose to use 32bit pointers etc. You can benefit from new ISA and more registers without 4Gb memory systems.
Making Smalltalk on ARM since 1986; making your Scratch better since 2012

tkaiser
Posts: 103
Joined: Fri Aug 05, 2016 1:28 pm

Re: 64-bit operating system

Sun Mar 25, 2018 7:20 pm

timrowledge wrote:
Sun Mar 25, 2018 7:11 pm
Not sure why I keep seeing assertions that 64bit code is bigger.
Have you seen this: https://github.com/nodesource/distribut ... -290440706 ?

Return to “General discussion”