Python one-liner statement

Merge two Python dictionaries in a single expression

For dictionaries x and yz becomes a merged dictionary with values from y replacing those from x.

  • In Python 3.5 or greater, :
    z = {**x, **y}
  • In Python 2, (or 3.4 or lower) write a function:
    def merge_two_dicts(x, y):
        z = x.copy()   # start with x's keys and values
        z.update(y)    # modifies z with y's keys and values & returns None
        return z

    and

    z = merge_two_dicts(x, y)

 Search in Python-list-of-dictionaries

>>> dicts = [
 { "name": "Tom", "age": 10 },
 { "name": "Mark", "age": 5 },
 { "name": "Pam", "age": 7 },
 { "name": "Dick", "age": 12 }
 ]
>>> next(item for item in dicts if item["name"] == "Pam")
{'age': 7, 'name': 'Pam'}

Remove-a-key-from-a-python-dictionary

By dict.pop():

my_dict.pop('key', None)

This will return my_dict[key] if the key exists in the dictionary, and None otherwise. If the second parameter is not specified (ie. my_dict.pop(‘key’)) and key does not exist, a KeyError is raised.

Advertisements

Sync data from MongoDb to ElasticSearch

mongo-connector1. mongo-connector:
mongo-connector is a real-time sync service as a package of python, which is a generic connection system that you can use to integrate MongoDB with another system with simple CRUD operational semantics. It creates a pipeline from a MongoDB cluster to one or more target systems, such as Solr, Elasticsearch, or another MongoDB cluster.

mongo-connector needs mongo to run in replica-set mode, sync data in mongo to the target then tail the mongo oplog, keeping up with operations in MongoDB in real-time. It needs a package named “elastic2_doc_manager” to write data to ES.

mongo-connector copies your documents from MongoDB to your target system. Afterwards, it constantly performs updates on the target system to keep MongoDB and the target in sync. The connector supports both Sharded Clusters and standalone Replica Sets, hiding the internal complexities such as rollbacks and chunk migrations.

Reference link is : mongo-connector

2. elasticsearch-river-mongodb:
Elasticsearch provides ability to enhance the basic functionality by plugins, which are easy to use and develop. They can be used for analysis, discovery, monitoring, data synchronization and many others. Rivers is a group of plugins used for data synchronization between database and elasticsearch.

There is a mongoDB river plugin for data synchronization, named “elasticsearch-river-mongodb”.

When document is inserted to MongoDB, database is created (if it doesn’t exist), along with schema for that particular record. Then, our data is stored. When more data comes in, the schema is updated. After inserting document in MongoDB configured as replica set, it is also stored in oplog collection.The mentioned collection is operations log configured as capped collection, which keeps a rolling record of all operations that modify the data stored in databases.

River plugin monitors this collection and forwards new operations to elasticsearch according to its configuration. That means that all insert, update and delete operations are forwarded to elasticsearch automatically. Missing index with default configuration was created automatically while indexing data in ES.

Reference link is: elasticsearch-river-mongodb

Web Performance Technique

Web performance is nothing but the speed in which web pages are downloaded and displayed on the user’s web browser. Faster website speeds have been shown to increase sales or increase visitor loyalty including user satisfaction. Particularly useful for those use slow internet connections or on mobile phones/tablets. You can increase the speed of page load using:

    • Server-side cache
    • Browser-side cache
    • Faster disks
    • Image optimization
    • Web application accelerator focusing on optimizing caching and compression
    • Load balancing and SSL offloading
    • Geo or DNS-based load balancing and more.

    In short, one can increase the time it takes for pages to render using multi-layered cache and asynchronous communication with server-side components

    Ref:
    Open Source Web Performance Software

Regex Patterns

To search for occurrences of more than one space between words in a line

[ ]{2,}

SPACE (2 or more)

You could also check that before and after those spaces words follow. (not other whitespaces like tabs or newlines)

\w[ ]{2,}\w

the same, but you can also pick (capture) only the spaces for tasks like the replacement

\w([ ]{2,})\w

or see that before and after spaces there is anything, not only word characters (except whitespace)

[^\s]([ ]{2,})[^\s]

Best general SVN Ignore Pattern?

I’ll add my own two cents to this question:

I use the following SVN ignore pattern with TortoiseSVN and Subversion CLI for native C++, C#/VB.NET, and PERL projects on both Windows and Linux platforms. It works well for me!

Formatted for copy and paste:

*.o *.lo *.la #*# .*.rej *.rej .*~ *~ .#* .DS_Store thumbs.db Thumbs.db *.bak *.class *.exe *.dll *.mine *.obj *.ncb *.lib *.log *.idb *.pdb *.ilk *.msi* .res *.pch *.suo *.exp *.*~ *.~* ~*.* cvs CVS .CVS .cvs release Release debug Debug ignore Ignore bin Bin obj Obj *.csproj.user *.user *.generated.cs

Formatted for readability:

*.o *.lo *.la #*# .*.rej *.rej
.*~ *~ .#* .DS_Store thumbs.db 
Thumbs.db *.bak *.class *.exe *.dll
*.mine *.obj *.ncb *.lib *.log 
*.idb *.pdb *.ilk *.msi* .res *.pch *.suo 
*.exp *.*~ *.~* ~*.* cvs  CVS .CVS .cvs  
release Release debug Debug
ignore Ignore bin Bin obj  Obj
*.csproj.user *.user
*.generated.cs

http://linqtovfp.codeplex.com/

http://randomdevnotes.com/2011/04/23/accessing-vfp-data-using-linqpad/

http://randomdevnotes.com/2009/08/19/linq-to-vfp-example-1/

http://randomdevnotes.com/2009/08/25/linq-to-vfp-example-2/

http://randomdevnotes.com/2009/09/09/linq-to-vfp-example-3/

 

http://fastdbf.codeplex.com/

To download The ‘VFPOLEDB.1’ provider  on the local machine.

http://www.microsoft.com/downloads/details.aspx?FamilyId=E1A87D8F-2D58-491F-A0FA-95A3289C5FD4&displaylang=en