Couchbase - Clone Records and Update Document Key

Couchbase is one of the best NoSQL database. I have used this for one of the project and recently came across a scenario for which I had to do some research but straightforward solution was not available. So, I just thought of documenting various steps so that other can be benefited.

So, in my scenario, I had to select some X records from one couchbase bucket and migrate them to another bucket. This was really easy. I wrote a N1QL query, added some WHERE clauses, and thats, done! All the selected records were migrated from one bucket to another.
Here comes the complexity. Someone updated the requirements, and now I was asked to update the "key" of the document. I was searching, searching and searching for correct and quick solution but found that we cannot update document "key" using N1QL. So, now I had to find some work around, and this is what I came up with (someone may have better solution for this scenario):

1. Create a "tmp" bucket and migrate selected records to some "tmp" bucket.
2. Use "cbbackup" command to take backup of the data from "tmp" bucket.
3. Download and open "DB Browser for SQLite"
4. Click on "Open Database" button, go to the backup folder and select *.cbb file from the folder.
5. Note that, if you have clustered environment, then you will notice multiple folders in backup location. You will have to repeat these steps for each *.cbb file.
6. After importing *.cbb file in SQLite, you will see that there are two tables: cbb_meta and cbb_msg
7. We are interested in cbb_msg table
8. Execute a query to update "key" value, you can also use existing functions. For example, I want to replace only portion of the key:
9. Select records and see if your changes correct
10. Now click on "Write Changes" button so that changes are saved in *.cbb file
11. Repeat this process to update keys from all the *.cbb files
12. Now, flush the "tmp" bucket and import updated *.cbb files
13. Use "cbrestore" command to import the *.cbb file in "tmp" bucket
14. Finally, migrate records from "tmp" bucket to desired bucket
Yes, I know this is long process, but worked for me. Hope this will be helpful to at least some of you.

P.S. Click here to access my other posts.

Comments

Postman - Set Timeout / Think Time / Pause / Delay

Those who are involved in API or web service development should be knowing about Postman , it is one of the most popular tools to build API requests and test them. Collection Runner is one of the feature of Postman . You can create one or more requests and group them in Collection, and as name suggests, you can run the entire collection i.e. series or requests. What if you need to add "Think Time" or "Delay" or "Pause" between two requests? It is surely possible, here are some options: Using Collection Runner GUI This option will be applicable to all the requests in the collection In the Collection Runner window, enter value for Delay in milliseconds Using Command Line This option will be applicable to all the requests in the collection Newman is a comman line collection runner for Postman Command To Execute: newman run <collection-file-source> --delay-request [number] Click here to get the details about Newman Newman installation

How to Extract Values from Response Header in JMeter?

JMeter is a powerful tool for API testing. Let's say you are are writing test cases for one of your RESTful service; and you want to extract and validate the value returned as part of response header. It is little bit tricky to extract the value from Response Header in JMeter , but it is possible. For example, your RESTful service returns "ETag" in response header. When you look at the raw response data, the value is displayed something like this: ETag: 2666d92fa9ebf10250acdb235546f045 To exact value of this reaponse header in JMeter: Right click on your HTTP request, then add Post Processor element - Regular Expression Extractor Select Radio button - Main sample only Select Radio burron - Response Headers Type some name in Reference Name section - for example, eTagVariable Type this expression in Reference Expression section - ETag:\s+(.+) IMPORTANT: This expression will select pick the ETag response header parameter and select everything after colon bla

Setup OpenLDAP on MAC

macOS (Mac OS X or OS X) is the current series of Unix-based graphical operating systems developed and marketed by Apple Inc . designed to run on Apple's Macintosh computers ("Macs"). Within the market of desktop, laptop and home computers, and by web usage, it is the second most widely used desktop OS after Microsoft Windows . Recently, while working on one of my projects, there was a requiremnt to integrate our system with LDAP i.e. Lightweight Directory Access Protocol ( LDAP ). So, I wanted to try out some samples by installing OpenLDAP on my MAC. Initially, it looked streightforward but later I realized that there are multiple steps involved to get OpenLDAP up and running in my Mac. So, I thought of documenting various steps so that others can refer them. What do you need to begin? MAC Obviously you need MAC because these steps will work only for MAC (Sorry Windows users) Homebrew This is a package manager for macOS. Click here and follow the instructio

MuleSoft - Static IP Addresses and Multiple Workers

Did you just realize that your Mule Application requires horizontal scalling ? Well, thats easy - just go to CloudHub Runtime Manager, select your application and change the "Workers" count from 1 to either 2, 3 or 4. If numbers are grayed out, then you might have to adjust the "Worker Size", or purchase additional capacity from MuleSoft. So, what is the issue then? At the time of the blog post, based on Mule documentation, if you are using only 1 worker and if you want to apply static IP, then you can simply navigate to "Static IPs" section in Runtime Manager and allocate the static IP address. The issue is when you want to apply Static IP addresses for more than 1 workers for various reasons including IP whitelisting. Based on Mule documentation you cannot apply static IPs if you are using more than 1 worker. From Mule Documentation: "Static IPs are not supported for private IP addresses inside a CloudHub VPC and it is only supported for app

Sublime Text 3 - Pertty Format JSON

You must be looking for some easy options to pretty format JSON data. Well, there are many websites which provides this capability and you can pretty format JSON data in browser. I use Sublime Text 3 text editor, and I was trying to see if I can pretty format JSON within the text editor itself. You can follow these simple steps: Open Sublime Text 3 text editor If you are using MAC OS Press Command + Shift + P Then select "Install Package" Search for "Pretty JSON" Install If you are using Windows OS Press CTRL + Shift + P Then select "Install Package" Search for "Pretty JSON" Install Once the installation is complete, select JSON string If you are using MAC OS Press Command + Control + J If you are using Windows OS Press CTRL + ALT + J Thats it! Now you don't have to copy your JSON string from Sublime Text 3, paste in your browser, format it, then copy formatted JSON from browser and paste it back in your Sublime

Access GitHub Repositories with SourceTree and 2FA (2 Factor Authentication)

Many developers and organizations use GitHub as code repository. Similarly, many prefer to commit code in GitHub repositories using various commands. I personally think that using any kind of user interface for committing code and performing various GIT operations is much more simpler and productive. SourceTree is one such Atlassian product. SourceTree is a free Git client and provides beautiful GUI that offers a visual representation for various Git repositories. For added security, you can enable 2FA (i.e. 2 Factor Authentication) for your GitHub account. Configuring GitHub with SourceTree is very very simple. Here are the steps on MAC OS (steps on Windows OS are similar): If you have not enabled 2FA: Open SourceTree and go to Preferences Select Accounts Click on Add button to add account On the pop-up window, select Host as "GitHub" Select Auth Type as "Basic" Type your GitHub username (not email) Type your password Select Protocol as HTTPS

MuleSoft LDAP Connector With Example

As we all know, the L ightweight D irectory A ccess Protocol ( LDAP ) is an open, vendor-neutral, industry standard application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network. A common use of LDAP is to provide a central place to store usernames and passwords. This allows many different applications and services to connect to the LDAP server to validate users. This has a major benefit that allows a central place to update and change user passwords. With MuleSoft’s LDAP Connector , we can access and maintain directory information services over an IP network by connecting to any LDAP server. LDAP Connector is developed by MuleSoft's developer community and it is categorized as Community Connectors. Latest LDAP Connector documentation is available here . Unfortunately, the information and examples of this connector are limited and scattered all over the web. Today, I am trying to cover as many details as p

MuleSoft - JSON Schema Validation with Dynamic Schema Location

While working on a Mule Application project, if you a planning to validate JSON payload against some JSON schema , and more importantly if you want to pass schema location dynamically, then this post will be benificial for you. Mule provides JSON Schema Validator as out of the box feature. From mule documentation , evaluates JSON payloads at runtime and verifies that they match a referenced JSON schema. You can match against schemas that exist in a local file or in an external URI. If the validation fails, an exception is raised with feedback about what went wrong and a reference to the original invalid payload. Above code works but here is the catch, in my scenario, the schema location was dynamic. Let's say the schema location is present in the database. Once the schema location value is retrieved from database, it is stored in a flow variable. For simplicity, I am creating a flow variable with hard-coded value. In actual application, the flow variable value will be the va

Mule SFTP and PGP Encryption

Many companies use standard protocols such as FTP and SFTP to transfer files to external partner and receive files from external partners. Using FTP and SFTP provides simple to use and low cost platform for file exchange. In some business scenarios, companies may have to exchange sensitive information such as employee’s personal information, expense reports, payment information etc. Mule has the ability to encrypt a message payload, or part of a payload, using Pretty Good Privacy ( PGP ). PGP combines data compression and data encryption to secure messages. The compression reduces message transmission time between origin and destination. There are two scenarios that this document addresses: Using another party’s public key to encrypt a messages in a Mule application Using one’s own set of private and public keys so as to accept, and decrypt messages in a Mule application. What is Pretty Good Privacy (PGP)? Pretty Good Privacy ( PGP ) is a data encryption and decryption

Run JMeter Tests with Maven

In this article, I will be focusing on configuring JMeter with Maven but lets first understand some basics of JMeter and Maven. The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. These days, performance testing is very very important especially when the applications are targeting large number of users. There are many tools available in market, some are paid, some are free. Apache JMeter is one such free and open source software. Though JMeter's was initially developed for load testing web applications, it is now far more advanced. The biggest advantage of the JMeter is that it can do many things like performance and functional testing for web services, databases, FTPs or Web Servers, LDAP, JMS, trigger emails/notifications. Most of these features are implemented with plugins. JMeter is powerful, easy to install and use and FREE! It is a Java desktop application with simple us

Out Of Memory!

Search This Blog