Colabtools: is it possible to increase the ram in google colab?

[dwy904]

is it possible to increase the memory limit. 11G is too small. Can't really do much.

[dwy904]

any update on this one?

[craigcitro]

We don't have any support for changing the resource footprint of a managed runtime at this point.

You can always connect to a local backend.

[ritubebu]

Any idea how to increase memory size?

[dwy904]

any update on this one?

[ziatdinovmax]

@dwy904, in my experience the best way to "purchase" additional RAM and/or faster GPUs in Colab is by connecting (from Colab notebook) to a deep learning VM on GCP where you can adjust specs of your instance each time before launching it. I made a short video on this a few weeks ago: https://drive.google.com/file/d/1ijawhI6AuPXxWhLu8flkrHZPTu3o_zF0/view?usp=sharing

[RamsteinWR]

Just crash your session by using the whole of the 12.5 Gigs of RAM then the door opens where you can directly double your RAM to 25 Gigs in Colab.

[juanlp]

@RamsteinWR is right. Just click on "Get more RAM" and you would be able to work on a 25Gbs instance. Not sure if it is still free?

[dwy904]

@RamsteinWR Interesting, do you have to crash before you can have more RAM?

[Spiffy1]

@RamsteinWR Interesting, do you have to crash before you can have more RAM?

Yes. As for how, I was inspecting my data and it maxed out my allocated RAM.

[Suvi-dha]

True, happens with me all the time. My data is huge. After the 12.72 GB RAM is maxed out, I immediately get the prompt of crash and option to increase my RAM.

[RamziFsm]

Google Colab gives free 25 Gb as Ram space. But for my model I need 64 Gb. How can I increase the memory space? Can I buy the rest and How?

[ziatdinovmax]

In my experience, the easiest way to get more computational recourses
without leaving your Colab notebook is to create a GCP deep learning VM
with larger memory and more powerful GPU(s) and then connect to that
instance from the Colab notebook.

On Mon, Sep 9, 2019 at 6:49 AM RamziFsm [e-mail] wrote:

Google Colab gives free 25 Gb as Ram space. But for my model I need 64 Gb.
How can I increase the memory space? Can I buy the rest and How?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/googlecolab/colabtools/issues/253?email_source=notifications&email_token=AIFIU2ZDXTMSXZPLC7BXRZTQIYS3HA5CNFSM4FSC2FBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6HDKHQ#issuecomment-529413406,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIFIU23IU5CKFFTDQ32NL4LQIYS3HANCNFSM4FSC2FBA
.

>

"Intelligence is the ability to adapt to change." - Stephen Hawking.

[RamziFsm]

Thanks ziatdinovmax for reply,
Do you know any link or tutorial for doing that?

[ziatdinovmax]

Sure, see e.g.
https://blog.kovalevskyi.com/gce-deeplearning-images-as-a-backend-for-google-colaboratory-bc4903d24947?gi=9dfbe3fc0111

On Mon, Sep 9, 2019 at 7:09 AM RamziFsm [e-mail] wrote:

Thanks ziatdinovmax for reply,
Do you know any link or tutorial for doing that?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/googlecolab/colabtools/issues/253?email_source=notifications&email_token=AIFIU2ZZYNLIQWFHLKNVHBTQIYVFHA5CNFSM4FSC2FBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6HFDCA#issuecomment-529420680,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIFIU22BDJNQZZ5ATWCY7CDQIYVFHANCNFSM4FSC2FBA
.

>

"Intelligence is the ability to adapt to change." - Stephen Hawking.

[elsheikh21]

For people searching for the problem, I hope this might help you out
you can just load a very large dataset into the ram,

1. Download and Unzip a huge dataset
2. Read the dataset into a var
3. Colab will crash and show you a message asking if you want to use their High Ram Option
4. Click yes of course
   and voilà

[klazaj]

Just crashed Colab via this code:

d=[]
while(1):
  d.append('1')

Indeed, a message shows up, asking if you want to increase RAM usage. RAM is then upgraded to 35GB.

[VuongTuanKhanh]

Just crashed Colab via this code:

d=[]
while(1):
  d.append('1')

Indeed, a message shows up, asking if you want to increase RAM usage. RAM is then upgraded to 35GB.

Thanks in advance, a messagebox shows up and then I have 25.51GB of RAM

[umeshpant]

Just crashed Colab via this code:

d=[]
while(1):
  d.append('1')

Indeed, a message shows up, asking if you want to increase RAM usage. RAM is then upgraded to 35GB.

Thanks in advance, a messagebox shows up and then I have 25.51GB of RAM

Thank you !
you save my day

[Abhishek109]

Can you help me how to increase my RAM from 25.5GB to higher limit

[VuongTuanKhanh]

Can you help me how to increase my RAM from 25.5GB to higher limit

As far as I know: Nope. Believe me, I've tried a lot of different ways. Maybe Colab limits the Session to a maximum of 25Gb. Perhaps you want to optimize the solution rather than increasing the amount of RAM.

But you know what, I'm happy about that. Isn't 25.5Gb free a great thing ;) I think we can't ask for more

[Abhishek109]

Can you help me how to increase my RAM from 25.5GB to higher limit

As far as I know: Nope. Believe me, I've tried a lot of different ways. Maybe Colab limits the Session to a maximum of 25Gb. Perhaps you want to optimize the solution rather than increasing the amount of RAM.

But you know what, I'm happy about that. Isn't 25.5Gb free a great thing ;) I think we can't ask for more

thats very true. thank you. have a great year ahead.

[Abdessamad139]

Can you help me how to increase my RAM from 25.5GB to higher limit

Actually, if you chose to change the runtime type from GPU to TPU, you will get 35.35 Gb instead of 25.5Gb

[Abhishek109]

Thank you Chanaa will follow your advice by changing to TPU.
Can you kindly suggest (any other console) if i would like to work on
higher than 35.35 GB for faster output.
That would be highly appreciative, Thanks in advance.

With best regards
Abhishek

On Sun, Jan 19, 2020 at 11:53 AM Chanaa Abdessamad [e-mail]
wrote:

[image: Screenshot (2)]
https://user-images.githubusercontent.com/57695875/71618963-097fd100-2be8-11ea-9af3-a9a456d1d076.png

Can you help me how to increase my RAM from 25.5GB to higher limit

Actually, if you chose to change the runtime type from GPU to TPU, you
will get 35.35 Gb instead of 25.5Gb

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/googlecolab/colabtools/issues/253?email_source=notifications&email_token=ANYF5AYPVEO6C62XWGIAO2LQ6PWWZA5CNFSM4FSC2FBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJKKDAA#issuecomment-575971712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANYF5A2F56JGGQCOWTMFEKLQ6PWWZANCNFSM4FSC2FBA
.

[Keramatfar]

Just crash your session by using the whole of the 12.5 Gigs of RAM then the door opens where you can directly double your RAM to 25 Gigs in Colab.

Does not work any more.

[Keramatfar]

Can you help me how to increase my RAM from 25.5GB to higher limit

Actually, if you chose to change the runtime type from GPU to TPU, you will get 35.35 Gb instead of 25.5Gb

Does not show the message.

[Keramatfar]

Can you help me how to increase my RAM from 25.5GB to higher limit

As far as I know: Nope. Believe me, I've tried a lot of different ways. Maybe Colab limits the Session to a maximum of 25Gb. Perhaps you want to optimize the solution rather than increasing the amount of RAM.

But you know what, I'm happy about that. Isn't 25.5Gb free a great thing ;) I think we can't ask for more

The problem is that in my case that is 12.75 Gb.

[ziatdinovmax]

Colab Pro (which I am using) allows one to select between low (~12 GB) and
high (~25 GB) RAM for any notebook. Highly recommend.

On Mon, May 25, 2020 at 12:18 PM Keramatfar [e-mail]
wrote:

[image: Screenshot (2)]
https://user-images.githubusercontent.com/57695875/71618963-097fd100-2be8-11ea-9af3-a9a456d1d076.png
Can you help me how to increase my RAM from 25.5GB to higher limit

As far as I know: Nope. Believe me, I've tried a lot of different ways.
Maybe Colab limits the Session to a maximum of 25Gb. Perhaps you want to
optimize the solution rather than increasing the amount of RAM.

But you know what, I'm happy about that. Isn't 25.5Gb free a great thing
;) I think we can't ask for more

The problem is that in my case that is 12.75 Gb.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/googlecolab/colabtools/issues/253#issuecomment-633640474,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIFIU23WNLPZHDZEXDZMHTDRTKK6BANCNFSM4FSC2FBA
.

--
"Intelligence is the ability to adapt to change." - Stephen Hawking.

[aish-where-ya]

Even I get the message "View Runtime Logs" instead of "Get more RAM".
I am still using 12GB and not 25GB.
Please can somebody help with this?

[aish-where-ya]

Even I get the message "View Runtime Logs" instead of "Get more RAM".
I am still using 12GB and not 25GB.
Please can somebody help with this?

I think this issue needs to be re-opened @craigcitro

[masterzzzen]

Unfortunately, I think they closed the "loophole"

[YukiSakuma]

So this free ram upgrade doesn't work anymore just when I was trying to train my StyleGAN 2 model I can't get to the 2nd iteration because it always go out of memory, any tips to make it work I'm using a 512x512 input images and using a pretrained model to train but the RAM always go over the limit... I tried adjusting the batch size but no success

[bicmfcs]

@craigcitro I think this issue need to be re-opened.
I can't see get more ram when my session crash because of using all ram (which is about 12 gig).
we really can't do much .
any solution ?

[forkbabu]

Not able to trigger 25GB ram option anymore.

[craigcitro]

This is WAI; related: #1246

As mentioned there, resources are not guaranteed may vary over time. For more context, see: https://research.google.com/colaboratory/faq.html#resource-limits

[KishoreKonakanti]

I dont think "crashing session" will guarantee you higher RAM... it depends on the usage... Having used both free and pro versions, here are limits on RAM:

Free Version: * Regardless of your acclerator (GPU/TPU/CPU), "free" version gives you only *12.5 GB

With Pro, you need to change RAM type:
Navigate to Runtime and click "Change Runtime type" and select Runtime shape as "High RAM"... this should give you 35.5 GB

[KishoreKonakanti]

@YukiSakuma did you try smaller resolutions ? Can you please post your config?

[YukiSakuma]

@YukiSakuma did you try smaller resolutions ? Can you please post your config?

Config-e, and I can't lower the resolution because I am trying to train from a pretrained model that was trained on 512x512 images.

[karthiikselvam]

@craigcitro I think this issue need to be re-opened.
I can't see get more ram when my session crash because of using all ram (which is about 12 gig).
we really can't do much .
any solution ?

The thing is that google has now changed their policy and this trick doesn't work anymore. The only way to get more than 12 GB of RAM is to make a copy of earlier notebook which was alloted 25 GB RAM.
Use this https://colab.research.google.com/drive/155S_bb3viIoL0wAwkIyr1r8XQu4ARwA9?usp=sharing
copy to your drive and follow the instructions in the file.

[hafiz031]

@craigcitro I think this issue need to be re-opened.
I can't see get more ram when my session crash because of using all ram (which is about 12 gig).
we really can't do much .
any solution ?

The thing is that google has now changed their policy and this trick doesn't work anymore. The only way to get more than 12 GB of RAM is to make a copy of earlier notebook which was alloted 25 GB RAM.
Use this https://colab.research.google.com/drive/155S_bb3viIoL0wAwkIyr1r8XQu4ARwA9?usp=sharing
copy to your drive and follow the instructions in the file.

Didn't work. Any new ways?

[SBAESH]

@craigcitro I think this issue need to be re-opened.
I can't see get more ram when my session crash because of using all ram (which is about 12 gig).
we really can't do much .
any solution ?

The thing is that google has now changed their policy and this trick doesn't work anymore. The only way to get more than 12 GB of RAM is to make a copy of earlier notebook which was alloted 25 GB RAM.
Use this https://colab.research.google.com/drive/155S_bb3viIoL0wAwkIyr1r8XQu4ARwA9?usp=sharing
copy to your drive and follow the instructions in the file.

Didn't work. Any new ways?

You just need to save a copy of it in your drive (from File -> Save a copy in Drive) and then you have the 25G RAM. Apparently you need to copy the code from the other colab to this one then.

[hafiz031]

@SBAESH yes now it works, thanks!

[pearlie317]

@craigcitro I think this issue need to be re-opened.
I can't see get more ram when my session crash because of using all ram (which is about 12 gig).
we really can't do much .
any solution ?

The thing is that google has now changed their policy and this trick doesn't work anymore. The only way to get more than 12 GB of RAM is to make a copy of earlier notebook which was alloted 25 GB RAM.
Use this https://colab.research.google.com/drive/155S_bb3viIoL0wAwkIyr1r8XQu4ARwA9?usp=sharing
copy to your drive and follow the instructions in the file.

Thanks so much. I made it !!!

[Feilouce]

I dont think "crashing session" will guarantee you higher RAM... it depends on the usage... Having used both free and pro versions, here are limits on RAM:

Free Version: * Regardless of your acclerator (GPU/TPU/CPU), "free" version gives you only *12.5 GB

With Pro, you need to change RAM type:
Navigate to Runtime and click "Change Runtime type" and select Runtime shape as "High RAM"... this should give you 35.5 GB

Hi, I have buy colab pro, also click”change runtime type” but only have 25.5GB RAM, any suggestion?

[tasomaniac]

While working on your projects, you most probably have lots of data created by other data. This goes on an on and collects lots of garbage. You can do the following to save some RAM space very easily. Deleting ~4 dataframes allowed me to save 6GB of RAM after garbage collection.

import gc

# random operation
trainAttrX, testAttrX, trainY, testY = train_test_split(df['trainData'], df['testData'], test_size=0.25, random_state=111)

# after this I don't need `df` anymore.
del df
gc.collect()