SSD as virtual memory

As SSD speeds keeps increasing, i'm thinking about using them as virtual RAM, mainly to load a ton of stable diffusion models into GPU memory without impacting the ram.
You don't really write anything with SD so it basically acts as a 1600-2000MHZ ram stick while having a frickton more storage, able to load 7GB models in a bit over a second.

Any thoughts? am i fricking up?

Tip Your Landlord Shirt $21.68

DMT Has Friends For Me Shirt $21.68

Tip Your Landlord Shirt $21.68

  1. 2 weeks ago
    Anonymous

    install gentoo

  2. 2 weeks ago
    Anonymous

    *kills it after 2 week*

    • 2 weeks ago
      Anonymous

      >am i fricking up?
      Yes, because of latency. If you don't need latency, you don't need swap either, just store it as is.

      This too.

      • 2 weeks ago
        Anonymous

        only gets damaged from writes
        i need a ton of (essentially) read only, fast access memory, ram's too expensive, SSDs got that

  3. 2 weeks ago
    Anonymous

    There is so much wrong with this post I don't even know where to start.
    Let's just boil it down to three critical points for the sake of simplicity

    1.Flash memory is slow as frick compared to RAM. The only reason SSDs are blazingly fast is because of parallelism and buffering. Which, by today's standards, uses mostly your computer's RAM as secondary and or sometimes even as the primary buffer.

    2.flash chips aren't invincible. They will wear out and die pretty fast given your use case.

    3.And this is why I almost started laughing while reading your post, The SSD has not enough bandwidth to cover your fricking use case. Even your local DDR RAM is too slow to be shared with your graphics card in order to generate stable diffusion or large language models at acceptable speeds.
    For that, you need a dedicated GDDR RAM Located on your GPU.
    32GB GDDR5 is the sweet spot without breaking a bank for a normal consoomer.

    • 2 weeks ago
      Anonymous

      *GDDR RAM located near your GPU on the gfx card

    • 2 weeks ago
      Anonymous

      i'm not doing LLMs, i'm generating images
      it only needs to load the model and LoRa into virtual memory then spits out the result
      there should be essentially zero writes by SD, only reads which won't wear at the SSD

      • 2 weeks ago
        Anonymous

        Its called a page file idiot

        Gpu memory controller is at constant usage when generating images

        You would get faster results with pure cpu using openvino

        • 2 weeks ago
          Anonymous

          i had a dude monitor his virtual memory and it's 99.5% read and 0.5% write (31mb image output, likely, with 6.5gb model)

          its called memory hierarchy for a reason. cpu cache is faster than vram. vram is faster than ram. ram is faster than your ssd. what you're asking for is one big pagefile swap.

          read

    • 2 weeks ago
      Anonymous

      >They will wear out and die pretty fast given your use case.
      this bullshit yet again?

      • 2 weeks ago
        Anonymous

        Do I look like a Russian?
        translate from communist to human language please

    • 2 weeks ago
      Anonymous

      So which card has 32gb of vram dj hitler?
      The only budget friendly option is a used 3090 with 24gb, which sucks.
      Personally, I wonder why theres not more pc's designed like the deck or the mac studio where ram/vram are the same because its all one SOC accessing it.
      If we werent talking handheld form factor or apple greed we could easily have 128gb usable ram for local ai, which would shit on nvidias current top offerings so hard its insane.

  4. 2 weeks ago
    Anonymous

    Ssd memory is limited to how many millions of ready and write u can get out of it, using it as swap for heavy loads is OK matter of fact my nvme hit 7000 r/w pm, and I use that when building and compiling locally, but using it as full on ram will kill it in few months

    • 2 weeks ago
      Anonymous

      given it's basically only reads (no rearranging, and its space will be mostly empty so any rearranging won't be requiring restructuring), and for larger batches, you'll only need to read 7-ish gigabytes every other minute

  5. 2 weeks ago
    Anonymous

    Yeah, it's called a paging file. Moron.

    • 2 weeks ago
      Anonymous

      yes, that's what we call virtual memory, i'm talking about benefiting from the SSD's increasing speeds to use them as a midway between hard drive and ram
      i'm about to make a few AI instances on some cheapass second hand desktops and even 32GB of ram is like 36 bucks, and there's some 512GB SSDs with the same byte transfer rate for a bit over 50 bucks, so i can instead make the thing run with a $2 2gb ram stick and have just enough space to make the SSD a faster, much bigger ram interface

      something something economy of scale

      • 2 weeks ago
        Anonymous

        >i'm talking about benefiting from the SSD's increasing speeds to use them as a midway between hard drive and ram
        No, your thread title is literally "SSD as virtual memory." That's a paging file.
        Next topic.

        • 2 weeks ago
          Anonymous

          yes i fricking know what a paging file is you moron
          i'm asking if it's any good for that case, like, can SD properly use virtual memory? does it actually write a lot to it or is it nice and contained inside of the GPU's VRAM?

          • 2 weeks ago
            Anonymous

            SATA and NVME are significantly slower than DIMM next topic.

      • 2 weeks ago
        Anonymous

        The transfer is not even remotely comparable between ram and the fastest ssd.
        Plus there is no image diffusion task that is going to eat up more than a typical 16gb system ram.

        However back when I tried running SD with 8 gigs ram it did load the model using my page file. It does this automatically and it was slow as dogshit.

        If its a cheap computer without a

        • 2 weeks ago
          Anonymous

          A-anon? I think the AI got him, guys.

  6. 2 weeks ago
    Anonymous

    use some cheap optane drives for that, if you can find them, is the best for your usecase.

  7. 2 weeks ago
    Anonymous

    What we really need is a standard to govern AI cluster task regulation so we can have an AIB with like 160 pcie lanes that holds 40+ NVMe pcie x4 cards each with a Hailo 10H NVMe card which each have their own 8GB LPDDR4 memory module soldered on, and that can bust out 1600 tops with 320GB of LPDDR4 combined on 160 watts plus whatever the controller uses.

  8. 2 weeks ago
    Anonymous

    its called memory hierarchy for a reason. cpu cache is faster than vram. vram is faster than ram. ram is faster than your ssd. what you're asking for is one big pagefile swap.

  9. 2 weeks ago
    Anonymous

    swapon
    apply directly to the forehead

  10. 2 weeks ago
    Anonymous

    You'll just kill your SSD in a very short timeframe. Using SSD or EEPROM as RAM is just a highway to disaster.

    • 2 weeks ago
      Anonymous

      already said when i tried, the only writes were image outputs, reads are free

      • 2 weeks ago
        Anonymous

        reads are NOT free on stacked/3D nand.

        • 2 weeks ago
          Anonymous

          What datasheet actually specifies a read workload?

          • 2 weeks ago
            Anonymous

            >source: my ass

            3D nand (which is usually CTF instead of floating gate) is significantly more susceptible to read disturb, which requires rewriting hot path data to other parts of the flash.

          • 2 weeks ago
            Anonymous

            yet again no source provided

          • 2 weeks ago
            Anonymous

            https://www.cs.cmu.edu/~yixinluo/index_files/read-disturb-errors_dsn15.pdf
            >Abstract—NAND flash memory reliability continues to degrade
            >as the memory is scaled down and more bits are programmed per
            >cell. A key contributor to this reduced reliability is read disturb,
            >where a read to one row of cells impacts the threshold voltages
            >of unread flash cells in different rows of the same block. Such
            >disturbances may shift the threshold voltages of these unread cells
            >to different logical states than originally programmed, leading to
            >read errors that hurt endurance

            Or to make it completely clear to you:
            >where a read [..] impacts the threshold voltages of unread flash cells in different rows [..] leading to read errors that hurt endurance

            Read disturb is a well documented phenomenon, and you can find hundreds of papers describing it and attempts at mitigating it.

          • 2 weeks ago
            Anonymous

            yet again no source provided

            And just in case you don't think endurance means endurance, they define it in section 4.6 as
            >endurance (the number of P/E cycles at which the total error rate becomes too large, resulting in an uncorrectable failure)
            So yes, reading flash DOES damage it. And it is WORSE for denser flash (which stacked 3d nand is) because electron leakage now occurs in 3 dimensions, instead of just 2.

          • 2 weeks ago
            Anonymous

            [...]
            And just in case you don't think endurance means endurance, they define it in section 4.6 as
            >endurance (the number of P/E cycles at which the total error rate becomes too large, resulting in an uncorrectable failure)
            So yes, reading flash DOES damage it. And it is WORSE for denser flash (which stacked 3d nand is) because electron leakage now occurs in 3 dimensions, instead of just 2.

            That's not damage to the hardware, it's damage to the data. They might rewrite the data to another cell if they think the cell is going to become unreadable but it doesn't follow from that text that they actually do so.

        • 2 weeks ago
          Anonymous

          >source: my ass

        • 2 weeks ago
          Anonymous

          only on fragmented shit, if you do large files it won't

    • 2 weeks ago
      Anonymous

      This is supposedly what Optane was supposed to be. It’s a shame they cancelled it, it could have been built into some pretty bad ass Intel-branded performance motherboards but Intel’s a bunch of troons and Jeets with zero cool ideas nowadays.

  11. 2 weeks ago
    Anonymous

    Never forget himem.sys

    • 2 weeks ago
      Anonymous

      Never forget to just download more memory.

  12. 2 weeks ago
    Anonymous

    you cant hard fault the gpu memory, meaning you cant swap with a gpu.
    Are you moronic?
    Also that would be slow as frick.

  13. 2 weeks ago
    Anonymous

    Stop removing your trip

  14. 2 weeks ago
    Anonymous

    NVME 5.0 = single channel DDR2, MAYBE slow DDR3

    You're insane OP. Also like everyone else said this will nuke the lifespan of the SSD.

  15. 2 weeks ago
    Anonymous

    This is CXL, it's a PCIe based protocol designed to allow the addition of system memory. Despite this, the chips used in said memory expander are not the same as normal SSDs, although they still are persistent.

    off topic, but this was an AVIF, not a webp or jpeg wtf

    • 2 weeks ago
      Anonymous

      Android phones are getting hardware acceleration for it so it's possible that this is going to be THE thing that actually replaces JPG.

      Unless AV1 flops and VVC takes over, then we're boned LOL.

      • 2 weeks ago
        Anonymous

        Patent restrictions are pretty much the #1 blocker for wider adoption for a media format, and is pretty much why AOM and AV1 exists in the first place. In this sense, VVC is already a flop. And also I'm not aware of an equivalent VVC image format like HEVC and AV1 have already.

        • 2 weeks ago
          Anonymous

          >Patent restrictions are pretty much the #1 blocker for wider adoption for a media format,
          Why does anyone say this shit when every device I own has h.265 decoding
          It's like saying royalties is what keeps HDMI from devices when it's literally the most manufactured display interface, even the first Pi had it when it was supposed to be the cheap computer

          • 2 weeks ago
            Anonymous

            Microsoft charges $1 for HEVC extensions, and nothing for AV1. I can imaging some B2B discounts being offered, but it's still there.

          • 2 weeks ago
            Anonymous

            1. most hardware decoders aren't licensed properly
            2. you'd have no software fallback and lots of people are running ancient devices without h265 hw decoding
            3. websites that host h265 encoded content would have to pay some fees
            >even first pi had it
            well the latest pi literally only has h265 decoding, doesn't even have h264 hw decoding, so it's safe to assume they're just israelites, not a good example
            also... the hdmi analogy makes no sense, you don't use hdmi to serve content on the web, and it doesn't affect compatibility with other people's devices you share stuff to

          • 2 weeks ago
            Anonymous

            >you'd have no software fallback and lots of people are running ancient devices without h265 hw decoding
            Yeah they'll just decode h264, yet another proprietary format
            >the hdmi analogy makes no sense
            Makes perfect sense
            Patents, royalties and other boogeyman doesn't have nearly the effect on adoption as IQfy tards like to think it does

  16. 2 weeks ago
    Anonymous

    Can you use you Google Drive account instead?

  17. 2 weeks ago
    Anonymous

    So sad actually that SSDs die so fast. Otherwise they are like the perfect storage tech

    • 2 weeks ago
      Anonymous

      SSDs will always outlive spinning rust due to no moving part. If a SSD dies fast on you its because it was heavily abused as swap or it has a dogshit controller.

    • 2 weeks ago
      Anonymous

      If there's anything to complain about, it's price for high-capacity SSDs. 8TB and more still cost an arm and a leg. Prices will eventually go down yes, but I'd rather see it now

  18. 2 weeks ago
    Anonymous

    but why? RAM is cheaper.

  19. 2 weeks ago
    Anonymous

    xd
    bro SSD is super turbo slow in comparison to cpu

  20. 2 weeks ago
    Anonymous

    >it basically acts as a 1600-2000MHZ ram
    lol, lmao

  21. 2 weeks ago
    Anonymous

    you can do it, but it will be slower than you think
    >it basically acts as a 1600-2000MHZ ram stick
    lol you wish
    in terms of latency ssd's are thousands of times slower than ram
    plus you seem to be greatly underestimating ram speed, you think 1600-2000MHz is 1.6-2.0GiB/s? the slowest DDR[1]-200 could do 1.6GiB/s, you aren't taking into account the bus width and the fact it's /ddr/ (two transfers per clock)

    • 2 weeks ago
      Anonymous

      https://i.imgur.com/fUO3o6v.jpeg

      As SSD speeds keeps increasing, i'm thinking about using them as virtual RAM, mainly to load a ton of stable diffusion models into GPU memory without impacting the ram.
      You don't really write anything with SD so it basically acts as a 1600-2000MHZ ram stick while having a frickton more storage, able to load 7GB models in a bit over a second.

      Any thoughts? am i fricking up?

      oh also, since they're not directly addressable by the cpu, any accesses to data in swap must be first copied into ram, so even if the ssd was the same speed, it's halved from that as well

Your email address will not be published. Required fields are marked *