Open source database porting tool idea
I have an idea for an open source tool that I'd like to help build, but I would first like to bounce it off of some folks before starting the journey.
The Problem:
With many different environments for a team, databases can rapidly become out of sync with each other (for good reason, data sets may be intentionally different from one another). The pain points that come with unique databases are these differences/idiosyncrasies and the "test-ability" of them when data specific issues are occurring. In the current world (or at least, my current world), it is necessary to dump databases and then manually create new ones from said backups. This action avoids corrupting the live database that other developers may still be using (or other internal customers -- you wouldn't want to touch them). Ideally, there would be an application to dump, copy and reinstantiate the database for someone to test on locally!
The Idea (I will use aws terms as I am most familiar with this cloud setup):
Build a database container launching tool that is given credentials to dump databases and export them to an s3 bucket. After a successful dump and copy, it will download the data and spin up a container with the selected database's data on it. I imagine this framework to wrap around kubernettes to easily "grab and go" with a new data set. Tap into your backups and do whatever testing you'd like!
Does this exist? Would it be useful?
database amazon-web-services kubernetes
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
add a comment |
I have an idea for an open source tool that I'd like to help build, but I would first like to bounce it off of some folks before starting the journey.
The Problem:
With many different environments for a team, databases can rapidly become out of sync with each other (for good reason, data sets may be intentionally different from one another). The pain points that come with unique databases are these differences/idiosyncrasies and the "test-ability" of them when data specific issues are occurring. In the current world (or at least, my current world), it is necessary to dump databases and then manually create new ones from said backups. This action avoids corrupting the live database that other developers may still be using (or other internal customers -- you wouldn't want to touch them). Ideally, there would be an application to dump, copy and reinstantiate the database for someone to test on locally!
The Idea (I will use aws terms as I am most familiar with this cloud setup):
Build a database container launching tool that is given credentials to dump databases and export them to an s3 bucket. After a successful dump and copy, it will download the data and spin up a container with the selected database's data on it. I imagine this framework to wrap around kubernettes to easily "grab and go" with a new data set. Tap into your backups and do whatever testing you'd like!
Does this exist? Would it be useful?
database amazon-web-services kubernetes
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
add a comment |
I have an idea for an open source tool that I'd like to help build, but I would first like to bounce it off of some folks before starting the journey.
The Problem:
With many different environments for a team, databases can rapidly become out of sync with each other (for good reason, data sets may be intentionally different from one another). The pain points that come with unique databases are these differences/idiosyncrasies and the "test-ability" of them when data specific issues are occurring. In the current world (or at least, my current world), it is necessary to dump databases and then manually create new ones from said backups. This action avoids corrupting the live database that other developers may still be using (or other internal customers -- you wouldn't want to touch them). Ideally, there would be an application to dump, copy and reinstantiate the database for someone to test on locally!
The Idea (I will use aws terms as I am most familiar with this cloud setup):
Build a database container launching tool that is given credentials to dump databases and export them to an s3 bucket. After a successful dump and copy, it will download the data and spin up a container with the selected database's data on it. I imagine this framework to wrap around kubernettes to easily "grab and go" with a new data set. Tap into your backups and do whatever testing you'd like!
Does this exist? Would it be useful?
database amazon-web-services kubernetes
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
I have an idea for an open source tool that I'd like to help build, but I would first like to bounce it off of some folks before starting the journey.
The Problem:
With many different environments for a team, databases can rapidly become out of sync with each other (for good reason, data sets may be intentionally different from one another). The pain points that come with unique databases are these differences/idiosyncrasies and the "test-ability" of them when data specific issues are occurring. In the current world (or at least, my current world), it is necessary to dump databases and then manually create new ones from said backups. This action avoids corrupting the live database that other developers may still be using (or other internal customers -- you wouldn't want to touch them). Ideally, there would be an application to dump, copy and reinstantiate the database for someone to test on locally!
The Idea (I will use aws terms as I am most familiar with this cloud setup):
Build a database container launching tool that is given credentials to dump databases and export them to an s3 bucket. After a successful dump and copy, it will download the data and spin up a container with the selected database's data on it. I imagine this framework to wrap around kubernettes to easily "grab and go" with a new data set. Tap into your backups and do whatever testing you'd like!
Does this exist? Would it be useful?
database amazon-web-services kubernetes
database amazon-web-services kubernetes
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
asked Nov 14 '18 at 5:01
Chad Van De HeyChad Van De Hey
1,062817
1,062817
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53293452%2fopen-source-database-porting-tool-idea%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53293452%2fopen-source-database-porting-tool-idea%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
