Going beyond 32bit memory addresing limit.

Hi to all.

I have been reading several related articles realated with DBMS memory addresing limit on 32 bit architecture and I am quite confused with some concepts. I would like start with a few list of current assumptions that I have gotten from my reading.

1.- For obvious reasons there is a limit of 2^32 bytes on the addresable memory on the 32 bits architecture.

2.- There are some technologies that could be used to "break" this limit (PAE,AWE on windows and VLM on Linux) allowing a process to address up to 64GB (Depends of technology used and platform)

3.- I have been looking for DBMS that could use these features to address huge bunch of memory to be used on some memory structures (mainly buffers pools). At this moment I have found "how to" references on MSSQL and Oracle (for Windows and Linux) to do this.

4.- I have been looking this kind of documents about DB2. I have read among other: "The DB2 UDB Model", "DB2 memory and file cache performance tuning on Linux", the "Exploting Large Memories" and the "Administration Guide:Performance"

5.- The introduction of EM64T/AMD64 technologies and the availability of operating systems that could use these new technologies will lbreak the memory limit. Theorically DBMS running on this plataform also will break the memory limit.


I hope somebody could give me either some references about further information that I could review or even some clarification on the nexts general topics.

1.- The "DB2 UDB Memory Model" defines that the 4GB memory limit exists on all 32bit platforms but also explain that windows 2003 could use PAE/AWE to allow to DB2 to create one buffer pool up to 64GB; "Exploting Large Memories" shows some concepts and configurations related with this topic. After this reading I got some confused.
So, it is posible that DB2 running on windows could breal the limit?,
Is there any "how to" reference?,
Is there any way to use memory beyond of 4GB on Linux?

2.- Assuming that there is a way on DB2 to break the 4GB limit.
The configured buffer pool will be used as a tipical buffer pool?
Is there any implication or colateral behavior for the instance using this technology?

3.- Is IBM going to have an DB2 UDB WSE that could run on EM64T/AMD64 operating systems?, When is expected to be ready?, will this release break the memory limit?


Kind regards

References:
(http://www-128.ibm.com/developerwork...6qi/index.html)
(http://www-128.ibm.com/developerwork...dm-0509wright/)
(http://www-1.ibm.com/support/docview...=utf-8&lang=en)

edson.estrella-at-microsiga.com wrote:
> Hi to all.
>
> I have been reading several related articles realated with DBMS memory addresing limit on 32 bit architecture and I am quite confused with some concepts. I would like start with a few list of current assumptions that I have gotten from my reading.
>
> 1.- For obvious reasons there is a limit of 2^32 bytes on the addresable memory on the 32 bits architecture.
>
> 2.- There are some technologies that could be used to "break" this limit (PAE,AWE on windows and VLM on Linux) allowing a process to address up to 64GB (Depends of technology used and platform)
>
> 3.- I have been looking for DBMS that could use these features to address huge bunch of memory to be used on some memory structures (mainly buffers pools). At this moment I have found "how to" references on MSSQL and Oracle (for Windows and Linux) to do this.
>
> 4.- I have been looking this kind of documents about DB2. I have read among other: "The DB2 UDB Model", "DB2 memory and file cache performance tuning on Linux", the "Exploting Large Memories" and the "Administration Guide:Performance"
>
> 5.- The introduction of EM64T/AMD64 technologies and the availability of operating systems that could use these new technologies will lbreak the memory limit. Theorically DBMS running on this plataform also will break the memory limit.
>
>
> I hope somebody could give me either some references about further information that I could review or even some clarification on the nexts general topics.
>
> 1.- The "DB2 UDB Memory Model" defines that the 4GB memory limit exists on all 32bit platforms but also explain that windows 2003 could use PAE/AWE to allow to DB2 to create one buffer pool up to 64GB; "Exploting Large Memories" shows some concepts and configurations related with this topic. After this reading I got some confused.
> So, it is posible that DB2 running on windows could breal the limit?,
> Is there any "how to" reference?,
> Is there any way to use memory beyond of 4GB on Linux?
>
> 2.- Assuming that there is a way on DB2 to break the 4GB limit.
> The configured buffer pool will be used as a tipical buffer pool?
> Is there any implication or colateral behavior for the instance using this technology?
>
> 3.- Is IBM going to have an DB2 UDB WSE that could run on EM64T/AMD64 operating systems?, When is expected to be ready?, will this release break the memory limit?
>
>
> Kind regards
>
> References:
> (http://www-128.ibm.com/developerwork...6qi/index.html)
> (http://www-128.ibm.com/developerwork...dm-0509wright/)
> (http://www-1.ibm.com/support/docview...=utf-8&lang=en)
>
Any chance you can migrate to 64-bit os so you can just use DB2 64-bit
technology?

Larry Edelstein

wrote in message
news:921096009.1136414488819.JavaMail.wassrvr-at-ltsg was007.sby.ibm.com...
> Hi to all.
>
> I have been reading several related articles realated with DBMS memory
> addresing limit on 32 bit architecture and I am quite confused with some
> concepts. I would like start with a few list of current assumptions that
> I have gotten from my reading.
>
> 1.- For obvious reasons there is a limit of 2^32 bytes on the addresable
> memory on the 32 bits architecture.
>
> 2.- There are some technologies that could be used to "break" this limit
> (PAE,AWE on windows and VLM on Linux) allowing a process to address up to
> 64GB (Depends of technology used and platform)
>
> 3.- I have been looking for DBMS that could use these features to address
> huge bunch of memory to be used on some memory structures (mainly buffers
> pools). At this moment I have found "how to" references on MSSQL and
> Oracle (for Windows and Linux) to do this.
>
> 4.- I have been looking this kind of documents about DB2. I have read
> among other: "The DB2 UDB Model", "DB2 memory and file cache performance
> tuning on Linux", the "Exploting Large Memories" and the "Administration
> Guide:Performance"
>
> 5.- The introduction of EM64T/AMD64 technologies and the availability of
> operating systems that could use these new technologies will lbreak the
> memory limit. Theorically DBMS running on this plataform also will break
> the memory limit.
>
>
> I hope somebody could give me either some references about further
> information that I could review or even some clarification on the nexts
> general topics.
>
> 1.- The "DB2 UDB Memory Model" defines that the 4GB memory limit exists on
> all 32bit platforms but also explain that windows 2003 could use PAE/AWE
> to allow to DB2 to create one buffer pool up to 64GB; "Exploting Large
> Memories" shows some concepts and configurations related with this topic.
> After this reading I got some confused.
> So, it is posible that DB2 running on windows could breal the limit?,
> Is there any "how to" reference?,
> Is there any way to use memory beyond of 4GB on Linux?
>
> 2.- Assuming that there is a way on DB2 to break the 4GB limit.
> The configured buffer pool will be used as a tipical buffer pool?
> Is there any implication or colateral behavior for the instance using
> this technology?
>
> 3.- Is IBM going to have an DB2 UDB WSE that could run on EM64T/AMD64
> operating systems?, When is expected to be ready?, will this release break
> the memory limit?
>
>
> Kind regards
>

DB2 supports very large memory addressing with a 64-bit instance. You will
need a 64 bit OS and the appropriate DB2 distribution. 64-bit distributions
of DB2 are available for the appropriate versions of Windows, Linux, and
UNIX (AIX, Solaris, and HP/UX).

edson.estrella-at-microsiga.com wrote:

>> Any chance you can migrate to 64-bit os so you can
>> just use DB2 64-bit
>> technology?
>>
>> Larry Edelstein
>
> Unfortunately migrate to full 64bit platform is not an option because my
> company works with small and medium business that can´t invest more money
> on hardware.
>
> For those clients to acquire new equipment based on EM64T could be an
> option, this is the reason for the question of DB2 release for this
> technology.

EM64T/AMD64 (collectively "x86-64" or just "x64") is a full 64-bit platform.
DB2 running natively on this platform (both Linux and Windows) will be able
to access around 2^64 bytes which is approximately "lots, even by today's
standard."

If your clients have this hardware already, all that is required to take
advantage is to ensure you're running the native OS (the 32-bit versions of
Linux and Windows both run on this hardware, you need to ensure you're
running the 64-bit version of Linux or Windows), and the native version of
DB2 for x86-64.

> Any chance you can migrate to 64-bit os so you can
> just use DB2 64-bit
> technology?
>
> Larry Edelstein

Unfortunately migrate to full 64bit platform is not an option because my company works with small and medium business that can´t invest more money on hardware.

For those clients to acquire new equipment based on EM64T could be an option, this is the reason for the question of DB2 release for this technology.

Edson Estrella

> >
>
> DB2 supports very large memory addressing with a
> 64-bit instance. You will
> need a 64 bit OS and the appropriate DB2
> distribution. 64-bit distributions
> of DB2 are available for the appropriate versions of
> Windows, Linux, and
> UNIX (AIX, Solaris, and HP/UX).
>

Including EM64T based architecture (Linux/Windows)?
Edson Estrella

> edson.estrella-at-microsiga.com wrote:
>
> >> Any chance you can migrate to 64-bit os so you can
> >> just use DB2 64-bit
> >> technology?
> >>
> >> Larry Edelstein
> >
> > Unfortunately migrate to full 64bit platform is not
> an option because my
> > company works with small and medium business that
> can´t invest more money
> > on hardware.
> >
> > For those clients to acquire new equipment based on
> EM64T could be an
> > option, this is the reason for the question of DB2
> release for this
> > technology.
>
> EM64T/AMD64 (collectively "x86-64" or just "x64") is
> a full 64-bit platform.
> DB2 running natively on this platform (both Linux and
> Windows) will be able
> to access around 2^64 bytes which is approximately
> "lots, even by today's
> standard."
>
> If your clients have this hardware already, all that
> is required to take
> advantage is to ensure you're running the native OS
> (the 32-bit versions of
> Linux and Windows both run on this hardware, you need
> to ensure you're
> running the 64-bit version of Linux or Windows), and
> the native version of
> DB2 for x86-64.
>
>


Thanks Daring this is one of the answers I have been looking for. Just for the record, based on your feedbak I looked into IBM DB2 web pages and found information supporting this topic:
http://www-306.ibm.com/software/data...tion-wsue.html
http://www-306.ibm.com/software/data...inux/validate/
http://www-306.ibm.com/software/data...b/sysreqs.html

Kind regards.

If you have more memory than can be addressed by a 32-bit instance,
here's how to use it, in order of preference:

1. create a 64-bit instance and have large bufferpool(s) and sortheaps
2. run EEE / DPF with a partitioned database, so each instance can
address part of the memory
3. devote the extra memory to filesystem caching (very useful if you
have LOBs on file system based tablespaces)
4. try DB2 techniques to address high memory like AWE and eStore.

Note: eStore will not be supported in the future, as the whole world
goes to 64-bit.

Note - in late 2005, even Micosoft stated that their future plans for
servers are 64-bit almost exclusively:

http://www.vnunet.com/itweek/news/21...ers-face-64bit

http://www.vnunet.com/vnunet/news/21...s-bit-strategy

edson.estrella-at-microsiga.com wrote:

>>edson.estrella-at-microsiga.com wrote:
>>
>>
>>>>Any chance you can migrate to 64-bit os so you can
>>>>just use DB2 64-bit
>>>>technology?
>>>>
>>>>Larry Edelstein
>>>
>>>Unfortunately migrate to full 64bit platform is not
>>
>>an option because my
>>
>>>company works with small and medium business that
>>
>>can´t invest more money
>>
>>>on hardware.
>>>
>>>For those clients to acquire new equipment based on
>>
>>EM64T could be an
>>
>>>option, this is the reason for the question of DB2
>>
>>release for this
>>
>>>technology.
>>
>>EM64T/AMD64 (collectively "x86-64" or just "x64") is
>>a full 64-bit platform.
>>DB2 running natively on this platform (both Linux and
>>Windows) will be able
>>to access around 2^64 bytes which is approximately
>>"lots, even by today's
>>standard."
>>
>>If your clients have this hardware already, all that
>>is required to take
>>advantage is to ensure you're running the native OS
>>(the 32-bit versions of
>>Linux and Windows both run on this hardware, you need
>>to ensure you're
>>running the 64-bit version of Linux or Windows), and
>>the native version of
>>DB2 for x86-64.
>>
>>
>
>
>
> Thanks Daring this is one of the answers I have been looking for. Just for the record, based on your feedbak I looked into IBM DB2 web pages and found information supporting this topic:
> http://www-306.ibm.com/software/data...tion-wsue.html
> http://www-306.ibm.com/software/data...inux/validate/
> http://www-306.ibm.com/software/data...b/sysreqs.html
>
> Kind regards.
>
>

wrote in message
news:778767534.1136474775865.JavaMail.wassrvr-at-ltsg was007.sby.ibm.com...
>
> Including EM64T based architecture (Linux/Windows)?
> Edson Estrella

Yes and also the AMD 64 chips. The new AMD 64 X2 dual core chips (and Intel
dual cores) are especially good since they DB2 only considers them to be one
CPU for licensing purposes.

The AMD 64 chips run much cooler than the equivalent Intel chips (at least
for now), and I would highly recommend them. IBM sells a complete DB2/AMD
turnkey system for parallel queries, so DB2 64 bit is fairly well tested on
AMD chips.
http://www-306.ibm.com/software/data/db2/linux/ice/