A collection of information technology and consulting knowledge
Category: Information Systems
Information Systems is an academic study of systems with a specific reference to information and the complementary networks of hardware and software that people and organizations use to collect, filter, process, create and also distribute data. … Information systems are also different from business processes.
This is one of those data type conversions which I
occasionally have to do in DataStage, but can never seem to remember. So, I
thought I would write this quick post to document the data type conversion code, which is
really easy, once I finally remember how to do it again.
The TimestampFromDateTime Function
I use the TimestampFromDateTime function to do this data type conversion. I’m sure there are other ways to achieve the result, but I find this method clean and easy to perform. The TimestampFromDateTimefunction is in the Functions > Date & Time menu.
To populate the function, you need only add your date field on
use ’00:00:00’ as your time element
TimestampFromDateTime Function Format
TimestampFromDateTime with Time Element example
TimestampFromDateTime(<<Date Field Here>>, ’00:00:00′)
Cloud computing is a service driven model for enabling ubiquitous, convenient, on demand network access to a shared pool computing resources that can be rapidly provisioned and released with minimal administrative effort or service provider interaction.
Today, newfound efficiencies and innovation are key to any business success – small, medium or large. In the rapidly evolving field of data analytics, innovative approaches to handling data are particularly important since data is the most valuable resource any business can have. IBM common SQL Engine is delivering application and query compatibility that is allowing companies to turn their data into actionable insights. This is allowing businesses to unleash the power of their databases without constraints.
But, is this really important?
Yes. Many businesses have accumulated tons of data over the years. This data resides in higher volumes, more locations throughout an enterprise – on-premise and on-cloud –, and in greater variety. Typically, this data should be a huge advantage, providing enterprises with actionable insights. But, often, this doesn’t happen.
IBM Hybrid Data Management.
With such a massive barrel of complex legacy data, many organizations find it confusing to decide what to do with it. Or where to start. The process of migrating all that data into new systems is simply a non-starter. As a solution, enterprises are turning to IBM Db2 – a hybrid, intuitive data approach that marries data and analytics seamlessly. IBM Db2 hybrid data management allows flexible cloud and on-premises deployment of data.
However, such levels of flexibility typically require organizations to rewrite or restructure their queries, and applications that will use the diverse, ever-changing data. These changes may even require you to license new software. This is costly and unfeasible. To bridge this gap, the Common SQL Engine (CSE) comes into play.
How IBM Common SQL Engine is Positioning Db2 for the Future?
The IBM Common SQL Engine inserts a single layer of data abstraction at the very data source. This means that, instead of migrating the data all at once, you can now apply data analytics wherever the data resides – whether on private, public or hybrid cloud – by using the Common SQL Engine as a bridge.
The IBM’s Common SQL Engine provides portability and consistency of SQL commands, meaning that the SQL is functionally portable across multiple implementations. It allows seamless movement of workloads to the cloud and allows for multiplatform integration and configurations regardless of their programming language.
Ideally, the Common SQL Engine is supposed to be the heart of the query and the foundation of application compatibility. But it does so much more!
Its compatibility extends beyond data analytic applications to include security, management, governance, data management, and other functionalities as well.
How does this improve the quality, flexibility, and portability of Db2?
By allowing for integration across multiple platforms, workloads and programming languages, the Common SQL Engine, ultimately, leads to a “data without limits” environment for Db2 hybrid data management family through:
Query and application compatibility
The Common SQL engine (CSE) ensures that users can write a query, and be confident that it will work across the Db2 hybrid data management family of offerings. With the CSE, you can change your data infrastructure and location – on-cloud or on-premises – without having to worry about license costs and application compatibility.
Data virtualization and Integration
The common SQL engine has a built-in data virtualization service that ensures that you can access your data from all your sources. These services position Db2 family of offerings including, IBM Db2 warehouse, IBM Db2, IBM Db2 BigSQL amongst others.
This services also applies to IBM Integrated Analytics System, Teradata, Oracle, Puredata and Microsoft SQL server. Besides, you can work seamlessly with open-source solutions such as HIVE; and cloud sources such as Amazon Redshift. Such levels of integration are unprecedented!
By allowing users to effectively pull data from Db2 data stores and integrate it with data from non-IBM stores using a single query, the common SQL engine places Db2 at an authoritative position as compared to other data stores.
Licensing is one of the hardest nuts to crack, especially for smart organizations who rely on technologies such as the cloud to deliver their services. While application compatibility and data integration will save you time, flexible licensing saves you money, on the spot.
IBM’s common SQL engine allows flexible licensing, meaning that you can purchase one license model and deploy it whenever needed, or as your data architecture evolves. Using IBM’s FlexPoint licensing, you can purchase FlexPoints and use them across all Db2 data management offerings. This is a convenience in one place.
The flexible licensing will not only simplify the adoption and exchange of platform capabilities, but it also positions your business strategically by making it more agile. Your data managers will be able to access the tools needed on the fly, without going through a lethargic and tedious procurement process.
IBM Db2 Data Management Family Is Supported by Common SQL Engine (CSE) .
IBM Db2 is a family of custom, deployable database that allows enterprises to leverage existing investments. IBM Db2 allows businesses to use any type of data from an either structured or unstructured database (or data warehouse). It provides the right data foundation/environment with industry-leading data compression, on-premise and cloud deployment options, modern data security, robust performance for mixed loads and the ability to adjust and scale without redesigning.
The IBM Db2 family enable businesses to adapt, scale quickly and remain competitive without compromising security, risk levels or privacy. It features:
Deployment and flexibility: On-premises, scale-on demand, and private or cloud deployments• Compression and performance
Embedded IoT technology is allowing businesses to act fast on the fly.
Some of these Db2 family offerings that are supported by the common SQL engine include:
Db2 Big SQL
Db2 on Cloud
Db2 Warehouse on Cloud
IBM Integrated Analytics System (IIAS)
Db2 Family Offerings and Beyond
Since the common SQL engine mainly focuses on data federation and propensity, other non-IBM databases can as well plug into the engine for SQL processing. These other 3rd party offerings include:
Watson Data Platform
Microsoft SQL Server
IBM Common SQL engine is allowing organizations to fully use data analytics to future-proof their business, and as well remain agile and competitive. In fact, besides the benefits of having robust tools woven into CSE, this SQL engine offers superior analytics and machine-learning positioning. Data processing can now happen at the speed of light –- 2X to 5X faster. The IBM Common SQL engine adds important capabilities to Db2, including freedom of location, freedom of use, and freedom of assembly.
Data analytics has changed where data is no longer manageable in relational databases only. Data is flowing from various sources which are not of the same format. This means it is not possible to store all data in the same repository. Some are best suited for storing in relational databases, others for Apache Hadoop while others are best suited for NoSQL databases.
During data analyzing, so much time is taken in trying to bring the distributed data together instead of obtaining insights. Db2 Federation has come to the rescue of data analysts. Federation concept in db2 eliminates the need for storing data in different repositories and reduces the hassle of getting insights.
What is DB2 Federation?
DB2 federation is a data integration technology that permits remote database objects to be accessed as local DB2 database objects. This technology connects multiple databases and makes them appear like one database.
How does DB2 federation work?
Federation allows you to access all of your data that is on multiple distributed databases using a single query. When implemented in an organization, this technology can be used to access data that is on any of the organization’s Db2, whether local or in the cloud.
Why use DB2 federation?
So, why should you use the federation? This concept brings data of all formats into one virtual source. With data being retrieved from one virtual source, analyzing it becomes cost-effective and efficient.
What are its primary use cases for DB2 federation?
Merging of various sources of data
DB2 federation facilitates consolidating of data from sources data local and cloud to form one virtual data source. This eliminates the process of migrating data which can be expensive and troublesome.
Increase the capacity of a repository beyond the fixed limits
Physical storage capacity is bound to have a limit which is one reason you may find an organization has distributed its data in various repositories. With federation, the storage is virtual and therefore doesn’t have any limit. This technology can greatly help you if your physical dataset is running low on space.
Linking up to Db2 Warehouse on Cloud
People who use Db2 products can federate data from Db2 on Cloud and Db2 Warehouse on the Cloud. This will give them a joint interface where they can access, add, query, and analyze data without encountering the complex ETL processes. Better still, no additional code will be required to execute all these processes. This makes it easy for people with the low technical know-how to use these products smoothly.
Split data across different servers
At times, you might choose to partition your data. With federation integration technology, partitioned data can be queried with a unified interface. Federation allows you to better balance your workloads, scale precise parts of an app, and create micro-services that work harmoniously.
Generally, db2 federation makes it access data by bringing it together into a single virtual source. This brings about cost and time-saving benefits. When you want to analyze data, you can get insights immediately instead of spending a lot of time querying through repositories.
A Road Map for Migrating to A Public Cloud Environment
Today, most organizations are looking for ways to cut down their sprawling IT budgets and define efficient paths for new developments. Making the move to the cloud is being seen as a more strategic and an economically viable idea, that is primarily allowing organizations to gain quick access to new platforms, services, and toolsets. But the migration of applications to the cloud environment needs a clear, and well-thought-out cloud migration strategy.
We are past the year of confusion and fear on matters cloud environment. In fact, almost everyone now agrees that the cloud is a key element of any company’s IT investment. What is not yet clear is what to move, how to move, and industry best practices to protect your investment in a public cloud environment. Therefore, a solid migration plan is an essential part of any cloud migration process.
Here are a few things you should pay close attention to when preparing a cloud migration planning template:
When planning to migrate to the cloud, it is paramount to remember that it is not a good idea to migrate every application. As you learn the baby steps, keep your legacy apps and other sensitive data such as private banking info off the cloud. This will ensure that, in case of a breach on your public cloud, your sensitive data and legacy systems will not fall into the hands of unsavory individuals.
Security of the data being migrated to the cloud should be just as important as on the cloud. Any temporary storage locations used during the cloud migration process should be secure from unauthorized intrusions.
Although security can be hard to quantify, it is one of the key components and considerations of any cloud service. The very basic security responsibility includes getting it right around your password security. Remember that, while you can massively increase the security around your applications, it is practically very different to deal with on-cloud threats and breaches since you technically don’t own any of the cloud software.
Some of the security concerns that you’ll need to look into include:
Is your data securely transferred and stored in the cloud?
Besides the passwords, does your cloud provider offer some type of 2-factor authentication?
How else are the users authenticated?
Does your provider meet the industry’s regulatory requirements?
Backup and Disaster recovery strategies
A backup and disaster strategy ensure that your data will be protected in case of a disaster. These strategies are unique to every organization depending on its application needs and the relevance of those applications to their organization.
To devise a foolproof DR strategy, it is important to identify and prioritize applications and determine the downtime acceptable for each application, services, and data.
Some of the things to consider when engineering your backup and disaster recovery blueprint include:
Availability of sufficient bandwidth and network capacity to redirect all users in case of a disaster.
Amount of data that may require backup.
Type of data to be protected
How long can it take to restore your systems from the cloud?
Communications Capacity enablement
Migrating to a cloud environment should make your business more agile and responsive to the market. Therefore, a robust communications enablement should be provided. Ideally, your cloud provider should be able to provide you with a contact center, unified messaging, mobility, presence, and integration with other business applications.
While the level of sophistication and efficiency of on-premise communications platforms depends on the capabilities of the company IT’s staff, cloud environments should offer communication tools with higher customizations to increase productivity.
A highly customized remote communications enablement will allow your company to refocus its IT resources to new innovation, spur agility, cut down on hardware costs and allow for more engagements with partners and customers.
Simply put, cloud communications:
Increase efficiency and productivity
Enables reimagined experience
Are designed for a seamless interaction.
legal liability and protection
Other important considerations when developing your cloud migration planning template are compliance with regulatory requirements and software licensing. For many businesses, data protection and regulatory compliance with HIPPA and GDPR is a constant concern, especially when dealing with identifiable data. Getting this right, the first time will allow you to move past the compliance issue blissfully.
When migrating, look for a cloud provider with comprehensive security assurance programs and governance-focused features. This will help your business operate more secure and in line with industry standards.
Ready to migrate your processes to a public cloud environment? Follow these pointers develop a comprehensive cloud migration planning template.
During the software engineering process, there are different issues which should be dealt with or else they will subject the project to unnecessary costs later. The technical debt perspective should be considered in each step of software development. For instance, when analyzing the cost of cloud approaches, you need to take into consideration the technical debt. You should as well factor the engineering aspect when making technical decisions such as choosing between cloud services vs. homegrown solutions.
What is technical debt?
Technical debt refers to the implied cost which will be incurred to do additional rework on a system after the engineering process is done. For example, engineers can choose to go for an easy option so that they can save time during the product design. The right steps which they will avoid will later need to be implemented which will mean a product has to be recalled or it will have to be fixed after it has reached the market which will cost more in terms of resources and manpower.
What are the most common types/causes technical debts?
Deliberate tech debt
In this case, engineers are aware of the step which is necessary during project implementation, but they will ignore it provided they can go for a shortcut which will save on cost and avail the product to the market. For instance, when analyzing the advantage of using the public cloud, some engineers may assume certain benefits, and later they will realize they are very necessary hence they are forced to go back and procure the system. It will lead to wastage in the company. Some engineers will not like doing the same process every now and then; they can avoid a given process only to expose the final product to flaws which will require re-engineering.
Accidental/outdated design tech debt
After designing a product or software, with time the technology will advance and render the design less effective in solving certain needs. For instance, due to advancement in technology, the tools you incorporated in a given software may end up being flawed which will make the product less effective which may necessitate re-engineering. Engineers may try their level best to come up with great designs, but advancement in technology can make their designs less effective.
Bit rot tech debt
It is a situation where a complexity develops over time. For example, a system or a component can develop unnecessary complexity due to different changes which have been incorporated over time. As engineers try to solve emerging needs, they can end up exposing the product to more complications which can be costly in the long run.
Strategies for minimizing technical debt
How to minimize deliberate tech debt
To avoid the tech debt, you need to track the backlog when engineers started the work. If you can track the backlog and identify areas where the engineers are trying to save time, you can avoid the debt.
Minimizing Accidental/outdated design tech debt
You need to refactor the subsystem every now and then so that you can identify the technical debt and fix it. For example, if the software is exposing you to unnecessary slowdowns, you need to fix the errors and make it meet industry standards.
Addressing Bit rot tech debt
Engineers should take time to understand the system they are running and clear any bad codes.
A public cloud strategy refers to a situation where you utilize cloud resources on a shared platform. Examples of shared or public cloud solutions include Microsoft Azure, Amazon Web Services and Google cloud. There are several benefits associated with cloud solutions. On the other hand, a private cloud strategy refers to a situation where you can decide to have an infrastructure which is dedicated to serving your business. It is sometimes referred to as homegrown where you employ experts to run the services so that your business can access different features. There are several advantages of using a public cloud over private cloud which you should know before you make an informed decision on the right platform to invest. Some of the benefits of the public cloud strategy include the following:
Availability and scale of Expertise
If you compare the public cloud and the private cloud services, the public cloud
allows you to access more experts. Remember the companies which offer the cloud services have enough employees who are ready to help several clients. In most cases, the other clients whom the service providers serve will not experience problems at the same time. It implies that human resource will be directed toward solving your urgent issue. You can as well scale up or down at any given time as the need arises which is unlike a case of private cloud solutions where you will have to invest in infrastructure each time you will like to upgrade.
Downgrading on a private cloud system can expose you to lose because you will leave some resources underutilized.
The volume of Technical Resources to apply
You access more technical resources in a public cloud platform. Remember the companies which offer the public cloud solutions are fully equipped with highly experienced experts. They also have the necessary tools and resources which
they can apply to assure you the best technical solutions each time you need them. It is unlike a private arrangement where you will have to incur more costs if the technical challenges will need advanced tools and highly qualified experts.
The price of a private cloud is high when compared to a public arrangement. If you are looking for ways you can save money, then the best way to go about it is to involve a public cloud solution. In the shared platform, you will only pay for
what you need. If you do not need a lot of resources at a given time, you can downgrade the services and enjoy fair prices. Services such as AWS offer great cost containment across the time which makes it easy to access the services at fair prices. For any business to grow, it should invest in the right package which brings the return on investment. The services offered by the public cloud systems allow businesses to save and grow. You should as well take into consideration other factors such as ecosystems for cloud relationships before you make an informed decision. There are some business models which prefer private cloud solutions while others can work well under public cloud-based solutions.