Or maybe it's here? "Where" meaning the right subforum here, or even entire different boards at other places. MOHO - mid sized home office. 
Ie stretching the definition of 'small' a bit but without a normal business level financial base.
Short version of my problem(s) - I am making plans to set up a fairly computer intensive scenario at one or more homes. The purpose is people working on film and video game projects in a form of networked studio. Were all students so were all poor-er but we can pool resources for essential things. I have people who know how to use certain software (or are willing to learn) and can produce the content, whether it's shooting video, editing or rendering 3d frames. The main problem is the hardware backend to all this. A previous attempt years ago ended in total and complete disaster across tens of terabytes due to a combination of silent data corruption, physically failing drives WAY before their claimed lifespan, haphazard backups, and a power surge coup de gras kicking dirt over and allowing flowers to grow.
So i'm trying to do things over now, but just do things right. However i'm to avoid creating a full time (or even part time) nightmare^B^B^B^B^B^B^ job for myself with the computer administration overhead since not only do I not want to be the IT Professional (nor can I afford to hire one, nor is one available willing to give time freely) I am busy working, going to college classes, AND working on some of the content myself.
Money is tight. But time including lost time by not having a system that works now is tighter. So the total plan involves figuring out what I can do now, and figuring out what I can migrate into later/not having data OR systems (including processes) unsuited to scale up. What might seem overengineered for this year might be because anything else would be inadequate 5 years down the road.
I am trying to do the R&D or learn about how to set up systems and processes (everything from the human side of backup and data migration strategies, decisions on fault tolerance thru disaster recovery, to hardware and software choices) once and for all that will scale up or down depending on any perceivable future needs. Figuring out a system of systems which mostly runs itself except when it sends me an email to replace an SSD or hard drive or failing HBA. Where I basically know what to do without thinking in event of any problem that occurs from a corrupt file to a total Disaster Recovery like the lightning strike that ended the last time. And where most problems have been prevented by doing proper system design in the first place. My main job afterwards should be down to buying hard drives/tapes to add to storage, replacing broken components, and sometimes adding or upgrading workstations to keep working. While following the instructions for any conceivable source of data loss and catastrophe without emotional stress or fear "we are ruined" again because The System is so robust it's already considered all that ahead of time. No more fears of what a virus or malware did, accidental deletings or editing the wrong file in an unrecoverable way.
One of the BIG things i'm wanting to do is to consider either migrating or setting up new hardware planned from the beginning to run diskless. Because i'm potentially facing DOZENS of computers running in just one house (and there will be potentially several others doing the same thing in their houses as a part of a VPN WAN) i'm thinking one of the simplest ways to reduce the overhead is to stop worrying about failing hard drives anymore. It's possible that set up in this way the total cost would even be cheaper (but i'm not sure) or better performing. Like a large block of RAM or SSDs caching to be used across multiple diskless computers (so each doesn't need it's own) only slowing down if too many are contending for it's performance. If there is a serious bottleneck for certain software though (video editing for instance) trying to run over normal 1GigE or a serious cost increase to handle it (going to 10GigE) i'd better reconsider. If most PC's are diskless and a few are not that's another okay compromise to start - I don't want to spend $300 for 10gig ethernet cards for instance until or unless it's needed at that station.
So there's my long explanation of what I have in mind, can anyone enlighten me to other terms or concepts I need to learn more about, maybe articles to read, or similar "enterprise style" solutions that could be attainable on smaller home office budgets, such as those enabled by using "obsolete" (to the mainstream business world) hardware which is still better than doing without, and which has a clear upgrade path into the future? For instance i'm interested in using fibre channel SANs with 4gig cards (cheap used now), maybe Infiniband at 10gig rate (same case), and last generation LTO6 tape drives on the data intensive back side in part because all of them have clear future migration strategies. Though i'd rather use standard gigabit ethernet for the majority of workstations simply because it's on everything and i'm hoping 120MB/sec shouldn't be a bottleneck in the majority of cases. (and spending $300 per HBA for 10gigE i'd rather avoid, since hard drives and tapes will be eating up alot of the budget afterwards)
Ie stretching the definition of 'small' a bit but without a normal business level financial base.Short version of my problem(s) - I am making plans to set up a fairly computer intensive scenario at one or more homes. The purpose is people working on film and video game projects in a form of networked studio. Were all students so were all poor-er but we can pool resources for essential things. I have people who know how to use certain software (or are willing to learn) and can produce the content, whether it's shooting video, editing or rendering 3d frames. The main problem is the hardware backend to all this. A previous attempt years ago ended in total and complete disaster across tens of terabytes due to a combination of silent data corruption, physically failing drives WAY before their claimed lifespan, haphazard backups, and a power surge coup de gras kicking dirt over and allowing flowers to grow.
So i'm trying to do things over now, but just do things right. However i'm to avoid creating a full time (or even part time) nightmare^B^B^B^B^B^B^ job for myself with the computer administration overhead since not only do I not want to be the IT Professional (nor can I afford to hire one, nor is one available willing to give time freely) I am busy working, going to college classes, AND working on some of the content myself.
Money is tight. But time including lost time by not having a system that works now is tighter. So the total plan involves figuring out what I can do now, and figuring out what I can migrate into later/not having data OR systems (including processes) unsuited to scale up. What might seem overengineered for this year might be because anything else would be inadequate 5 years down the road.
I am trying to do the R&D or learn about how to set up systems and processes (everything from the human side of backup and data migration strategies, decisions on fault tolerance thru disaster recovery, to hardware and software choices) once and for all that will scale up or down depending on any perceivable future needs. Figuring out a system of systems which mostly runs itself except when it sends me an email to replace an SSD or hard drive or failing HBA. Where I basically know what to do without thinking in event of any problem that occurs from a corrupt file to a total Disaster Recovery like the lightning strike that ended the last time. And where most problems have been prevented by doing proper system design in the first place. My main job afterwards should be down to buying hard drives/tapes to add to storage, replacing broken components, and sometimes adding or upgrading workstations to keep working. While following the instructions for any conceivable source of data loss and catastrophe without emotional stress or fear "we are ruined" again because The System is so robust it's already considered all that ahead of time. No more fears of what a virus or malware did, accidental deletings or editing the wrong file in an unrecoverable way.
One of the BIG things i'm wanting to do is to consider either migrating or setting up new hardware planned from the beginning to run diskless. Because i'm potentially facing DOZENS of computers running in just one house (and there will be potentially several others doing the same thing in their houses as a part of a VPN WAN) i'm thinking one of the simplest ways to reduce the overhead is to stop worrying about failing hard drives anymore. It's possible that set up in this way the total cost would even be cheaper (but i'm not sure) or better performing. Like a large block of RAM or SSDs caching to be used across multiple diskless computers (so each doesn't need it's own) only slowing down if too many are contending for it's performance. If there is a serious bottleneck for certain software though (video editing for instance) trying to run over normal 1GigE or a serious cost increase to handle it (going to 10GigE) i'd better reconsider. If most PC's are diskless and a few are not that's another okay compromise to start - I don't want to spend $300 for 10gig ethernet cards for instance until or unless it's needed at that station.
So there's my long explanation of what I have in mind, can anyone enlighten me to other terms or concepts I need to learn more about, maybe articles to read, or similar "enterprise style" solutions that could be attainable on smaller home office budgets, such as those enabled by using "obsolete" (to the mainstream business world) hardware which is still better than doing without, and which has a clear upgrade path into the future? For instance i'm interested in using fibre channel SANs with 4gig cards (cheap used now), maybe Infiniband at 10gig rate (same case), and last generation LTO6 tape drives on the data intensive back side in part because all of them have clear future migration strategies. Though i'd rather use standard gigabit ethernet for the majority of workstations simply because it's on everything and i'm hoping 120MB/sec shouldn't be a bottleneck in the majority of cases. (and spending $300 per HBA for 10gigE i'd rather avoid, since hard drives and tapes will be eating up alot of the budget afterwards)
Last edited:
