eCommerce Technologies and Infrastructures

Diomidis Spinellis
 Department of Management Science and Technology
Athens University of Economics and Business
Athens, Greece
dds@aueb.gr

Application architectures

Architectural Dimensions

Basic Design Principles

When designing a system's architecture we try to follow the principles outlined below.

Abstraction

Abstraction is achieved in the following areas:

Refinement

Stepwise refinement is a way to tame the details and complexity of the final design.

Modularity

A modular design minimises the system's complexity and thereby the development cost and the number of possible errors. A good design method should provide:

Cohesion

A design should bring closely together parts that exhibit a high measure of cohesion. We can distinguish the following ordered types of cohesion:
  1. coincidental cohesion
  2. logical cohesion
  3. temporal cohesion
  4. procedural cohesion
  5. communicational cohesion
  6. sequential cohesion
  7. functional cohesion

Coupling

On the other hand, a design should avoid coupling between parts. We can distinguish the following ordered types of coupling:
  1. data coupling
  2. stamp coupling (copy of a part)
  3. control coupling
  4. common coupling
  5. external coupling
  6. content coupling

A: System Structures

Important and noteworthy system structuring models include the following:

Centralized Repository

Examples

Data-Flow

Create Manual Page Index

Data flow diagram

Object-Oriented

A generalization relationship in the Tomcat servlet container

Generalization data flow diagram

Layered

The Windows NT implementation layers

Windows NT kernel structure

Hierarchies

Hierarchies

The NetBSD kernel source hierarchy

The NetBSD kernel source hierarchy

Control Models

Event-Driven Systems

System Manager

State Transition

TCP state transition diagram

TCP state transition diagram

B: Element Packaging Approaches

Elements can be packaged using the following approaches:

Module

Namespace

Object

Generic Implementation

Abstract Data Type

Examples

Library

Component

Process

Data Repository

C: Architectural Reuse

Reusing architectural designs is often more important than reusing code. Some commonly-used approaches are:

Framework

Code Wizard

Design Patterns

"Each pattern describes a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice."
- Christopher Alexander

Domain-Specific and Reference Architectures

D: Web Services Architecture

Web Service

Service oriented architecture

Web Services Architecture

Roles of Humans and Machines

Machines and humans interacting in WSA

Web Service Technologies

Technologies in the WSA stack

WSA Models

WSA model relationships

Message Oriented Model

ER diagram of the WSA Message Oriented Model

Important Definitions

Service Oriented Model

ER diagram of the WSA Service Oriented Model

Important Definitions

Resource Oriented Model

ER diagram of the WSA Resource Oriented Model

Important Definitions

Policy Model

ER diagram of the WSA Policy Model

Important Definitions

Management Model

ER diagram of the WSA Management Model

Important Definitions

Bibliography

Database systems

Advantages

Problems and Limitations

Levels of Abstraction

Physical schema
Defines how data is stored
Conceptual schema or logical schema
Defines data in terms of a data model
External schema or view level
Defines a number of simplified domain-specific views

DBMS Levels of Abstraction

DBMS Levels of Abstraction

Data Independence

The three different levels of abstraction allow us to:

Database Design

  1. Requirements analysis
  2. Conceptual database design: develop high level description
  3. Logical database design: map description into the specific DBMS
  4. Schema refinement: identify and solve potential problems
  5. Physical database design: optimize for specific workloads
  6. Security design

Entities and Attributes

Entity
Object in the real world
Attributes
Describe each entity
Entity set
Group of similar entities (share same attributes)
Attribute domain
Values that can be used for the attribute
Key
Minimal set of attributes that uniquely identify an entity
Primary key
Designated among a number of candidate keys

The books entity set

The books entity set

The Entity Relationship Model

Relationship
Association between two entities
Relationship set
Group of similar relationships
Descriptive attributes
can identify a relationsip
A relationship can be identified as:

The Published relationship

The Published relationship

The Relational Model

Relation Schema

Relation Instance

(Table of rows with the same number of fields)
ISBNTitlePrice
0-387-02620-7Beyond Fear29.00
0-201-79940-5Code Reading49.95
0-07-246535-2Database Management Systems40.00
0-19-502402-8The Timeless Way of Building60.00

Redundancy

Storing redundant information in a database results in:

Example

ISBNTitlePriceDiscountReduced Price
0-387-02620-7Beyond Fear30.001027.00
0-201-79940-5Code Reading50.002040.00
0-07-246535-2Database Management Systems40.002038.00
0-19-502402-8The Timeless Way of Building60.001054.00

Dependencies are resolved by decomposing the relations.

The Structured Query Language

Language used for relational DBMSs.

Used:

SQL Table Definition and Data Manipulation

Create the Table

CREATE TABLE Books (
        isbn CHAR(13),
        title CHAR(100),
        price real)

Add a Record

INSERT
INTO Books (isbn, title, price)
VALUES ("0-201-79940-5", "Code Reading", 49.95)

Modify Records

UPDATE Books B
SET B.price = B.price * 1.1

Delete a Record

DELETE
FROM Books B
WHERE ISBN="0-201-79940-5"

SQL Queries

Display the Details for a Record

SELECT *
FROM Books
WHERE ISBN="0-201-79940-5"

Display Records Based on a Condition

SELECT title, price
FROM Books
WHERE price > 50 AND price > 10

Perform an Aggregate Calculation

SELECT AVG(price)
FROM Books

Join Two Tables

SELECT title, authorname
FROM Books, Authors
WHERE Books.authorid = Authors.authorid

Query By Example

Query by Example in Microsoft Access
SELECT BOOKS.title, BOOKS.price, BOOKS.price, BOOKS.isbn
FROM BOOKS
WHERE (((BOOKS.price)>100)) OR (((BOOKS.price)<10))
ORDER BY BOOKS.title;

Additional SQL Query Constructs

SQL provides many sophisticated capabilities:

Views

By defining appropriate views on a schema we
CREATE VIEW BrowseBooks(isbn, title)
        AS SELECT B.isbn, B.title
        FROM Books B
        WHERE B.price < 100

Indexes

An index is an auxilliary data structure used to speed up operations that are not efficiently supported by a table's record organisation.
CREATE INDEX IndPrice ON Books
WITH STRUCTURE = BTREE
KEY = (price)
To design the index structure take into account:

Security Issues

Our goal in designing a secure database is to achieve: Discretionary access control provides us the capability to give (and revoke) rights to specific users or groups.

Examples

GRANT SELECT
ON BrowseBooks
TO WebUsers

REVOKE INSERT, DELETE
ON Books
From Alice

GRANT INSERT, DELETE
ON Books
TO InventoryGroup

GRANT UPDATE(price)
ON Books
TO MarketingGroup

GRANT UPDATE(title, isbn)
ON Books
TO MaintenanceGroup

Transaction Management

Four important properties (ACID):
Atomic
Database ensures that all actions are carried out, or none
Consistency
Users ensure that transactions leave the data in a consistent state
Isolation
Users to not have to worry about concurrently executing transactions
Durability
Completed transactions persist after a crash even if the database has not been updated

Crash Recovery

The Database Administrator

Coordinates all the activities of the database system; the database administrator has a good understanding of the enterprise's information resources and needs.

The database administrator's duties include:

User Profiles

Users are differentiated by the way they expect to interact with the system
Application programmers
interact with system through DML calls
Sophisticated users
form requests in a database query language
Specialized users
write specialized database applications that do not fit into the traditional data processing framework
Naive users
invoke one of the permanent application programs that have been written previously

RAID Storage

Redundant arrays of independent disks (RAID) are used to overcome two bottlenecks associated with disk storage: By using additional redundant disks we can increase both. The following are some typically identified RAID levels:
0: Nonredundant
Data is distributed across different disks to increase performance
1: Mirrored
A second disk set keeps a copy of the data (50% overhead)
0+1 (or 10): Striping and mirroring
Data is distributed across a second disk set to increase performance
2: Error correcting codes
Additional check disks (4:3, 10:4, 25:5) are used to provide redundancy with a smaller overhead (e.g. 57%, 71%, 83%)
3: Bit-interleaved parity
A single additional check disk is used to recover the data by distributing bits across all disks
4: block-interleaved parity
The check disk contains the parity on a block level: higher read throughput
5: block-interleaved distributed parity
Block parity is distributed across all disks: higher read and write throughput
6: higher redundancy
Like 5 with an additional check disk guarding against a second failure

XML Concepts

XML Syntax

An XML document is structured on the following building blocks:
Elements
Delimited by starting and ending tags
Example:
        <title>The Lord of the Rings</title>
Elements can also be empty; a special shortcut makes them smaller:
Example:
        <dvd></dvd>
is the same as
        <dvd />
Attributes
Name=valua pairs in the starting tag, that can be used for metadata
Example:
        <title lang="english">The Lord of the Rings</title>
Entity references
Are used to refer to special (reserved) symbols
SymbolEntity reference
<&lt;
>&gt;
&&amp;
"&quot;
'&apos;
Comments
Delimited by the <!-- and -> sequence
Example:
        <!-- this is a comment ->

Example

<!-- List of shipped books -->
<booklist>
        <book>
                <isbn>0-387-02620-7</isbn>
                <title>Beyond Fear</title>
                <author>
                        <givenname>Bruce</givenname>
                        <familyname>Schneier</familyname>
                </author>
                <price>30.00</price>
        </book>
        <book>
                <isbn>0-201-79940-5</isbn>
                <title>Code Reading</title>
                <author>
                        <givenname>Diomidis</givenname>
                        <familyname>Spinellis</familyname>
                </author>
                <price>50.00</price>
        </book>
        <book>
                <isbn>0-07-246535-2</isbn>
                <title>Database Management Systems</title>
                <author>
                        <givenname>Raghu</givenname>
                        <familyname>Ramakrishnan</familyname>
                </author>
                <author>
                        <givenname>Johannes</givenname>
                        <familyname>Gehrke</familyname>
                </author>
                <price>40.00</price>
        </book>
        <book>
                <isbn>0-19-502402-8</isbn>
                <title>The Timeless Way of Building</title>
                <author>
                        <givenname>Cristopher</givenname>
                        <familyname>Alexander</familyname>
                </author>
                <price>60.00</price>
        </book>
        <book>
                <isbn>3-8218-0479-3</isbn>
                <title lang="german">Lexikon der Populaeren Irrtuemer</title>
                <author>
                        <givenname>Walter</givenname>
                        <familyname>Kraemer</familyname>
                </author>
                <author>
                        <givenname>Goetz</givenname>
                        <familyname>Trenkler</familyname>
                </author>
                <price>40.00</price>
        </book>
</booklist>

Document Type Definitions

Example:
  <!ELEMENT booklist (book)*>
    <!ELEMENT book (isbn,title,author+,price,edition?)*>
      <!ELEMENT isbn (#PCDATA)>
      <!ELEMENT title (#PCDATA)>
      <!ATTLIST title lang (german|english) "german">
      <!ELEMENT author (givenname,familyname)>
        <!ELEMENT givenname (#PCDATA)*>
        <!ELEMENT familyname (#PCDATA)>
      <!ELEMENT price (#PCDATA)>
      <!ELEMENT edition (#PCDATA)>

Bibliography

System administration

Client Workstations

Main tasks:

Servers

Important elements:

Rack-mounted servers
Rack-mounted servers

Services

Basic principles:

Debugging Techniques

Basic strategy: Two basic techniques: Tool examples:

The Fix-Once Principle

Namespaces

Namespaces: Naming policies: Management policy:

Disaster Planning and Recovery

Disaster types budget = disaster cost * disaster probability

Actions:

Change Management

Basic principles:

RCS Log Example

RCS file: RCS/ppp.conf,v
Working file: ppp.conf
head: 1.10
branch:
locks: strict
        dds: 1.10
access list:
symbolic names:
keyword substitution: kv
total revisions: 10;    selected revisions: 10
description:
----------------------------
revision 1.10   locked by: dds;
date: 2002/01/21 09:27:15;  author: dds;  state: Exp;  lines: +5 -0
Added CVS pserver rule.
----------------------------
revision 1.9
date: 2001/12/21 07:47:57;  author: dds;  state: Exp;  lines: +3 -3
New phone numbers.
----------------------------
revision 1.8
date: 2001/12/13 07:46:12;  author: dds;  state: Exp;  lines: +6 -5
Disable multilink.
Allow NTP packets.
----------------------------
revision 1.7
date: 2001/11/22 14:44:25;  author: dds;  state: Exp;  lines: +20 -4
Added aueb connection support.
Fixed to work with outgoing ping.
Added missing traceroute roules.
----------------------------
revision 1.6
date: 2001/11/06 19:23:33;  author: dds;  state: Exp;  lines: +16 -6
Blocked SNMP
Contrained application-level data
Fixed cddb.
----------------------------
revision 1.5
date: 2001/11/06 19:22:24;  author: dds;  state: Exp;  lines: +17 -7
Added CDDB, application lelvel data, ...
----------------------------
revision 1.4
date: 2001/02/04 15:04:18;  author: dds;  state: Exp;  lines: +1 -2
Timeout to 2.5 minutes.
----------------------------
revision 1.3
date: 2001/02/03 11:16:06;  author: dds;  state: Exp;  lines: +1 -1
Do not enable DNS, we have our own.
----------------------------
revision 1.2
date: 2001/01/30 23:11:48;  author: dds;  state: Exp;  lines: +168 -11
Added hosts and filters.
----------------------------
revision 1.1
date: 2001/01/20 19:26:15;  author: dds;  state: Exp;
Initial revision
----------------------------
=============================================================================

Manual Change Log Example

Thu Dec 04 14:20:52 EST 2003
To correct wireless dhcp problems download and install fix KB826942

Tue Dec 24 10:16:13 EET 2002
Enable time serving for Windows clients in smb.conf
 time server = True
From Windows execute: net time /set

Thu Dec  5 09:24:38 EET 2002
build kernels
run mergemaster
make installkernel KERNCONF=SPITI
cp /vol/obj/vol/src/sys/OFFICE/kernel /usr/export/boot

Wed Dec  4 15:15:08 EET 2002
In /usr/src run build and install
co /sys/dev/ata/ata-disk.c
rebuild kernel

Mon Dec  2 12:50:17 EET 2002
In /vol/ports/databases/p5-DBD-SQLite/work/DBD-SQLite-0.21
run perl getsqlite.pl 2.7.3 after manually downloading the file)
After that make, make install

Thu Nov 28 10:02:53 EET 2002
pkg_delete newt-0.50.13 dhcpconf-0.7.0
pkg_delete tcl-8.2.3
make install in /vol/ports/databases/sqlite
(also installed tcl-8.3.5)

Sun Nov 17 13:21:29 EET 2002
Run cvsup -g -L 2 -z supfile
Run cvsup -g -L 2 -z ports-supfile
Install ports/p5-DBI
Install ports/p5-DBD-SQLite

Upgrades

Server upgrade procedure:
  1. Existing services
    1. Services offered by the server
    2. Service users
    3. Software offering the services
  2. Ensure each software element works in the new release
  3. Plan testing procedure for each service
  4. Create back-out plan
  5. Choose maintenance interval
  6. Notify users
  7. Run test plan on existing server
  8. Upgrade (with an assistant)
  9. Repeat tests
  10. If tests fail follow back-out plan
  11. Notify customers

Upgrading TAXISnet
Upgrading TAXISnet

Centralising and Decentralising

We often must decide for or against central or distributed approaches. The following facts can aid our decision: Centralised management candidates:

The Helpdesk Service

To plan a helpdesk service you need to consider: An important element of a helpdesk service is an information system that provides the following functionality:

Example of a Problem Entry Form (Bugzilla)

Product Browser
Component

The area where the problem occurs. To pick the right component, you could use the same one as similar bugs you found in your search, or read the full list of component descriptions if you need more help.

Select a component to see its description here.
Hardware Platform
Operating System
Build Identifier
(User Agent)

This is the line beginning "Mozilla/5.0" in Help | About Mozilla. It identifies exactly the version of Mozilla you were using. If you are using the problem copy of Mozilla, this will be filled in correctly for you already. If Mozilla won't start, just enter the URL of the directory you downloaded it from.

URL

URL that demonstrates the problem you are seeing (optional.) IMPORTANT: If you are filing a bug about a broken web page, you need to read the Mozilla Web Developer FAQ, which explains how we handle old or badly-coded web pages.

Summary

A sentence which summarises the problem. Please be descriptive and use lots of keywords.

Bad example: mail crashed
Good example: crash if I close the mail window while checking for new POP mail

Details

Expand on the Summary. Please be as specific as possible about what is wrong.

Bad example: This page looks wrong in Mozilla. Other browsers display it correctly. You suck!
Good example: In the page at http://www.foo.com/, text in the first column should be fully justified -- text should be aligned with both the left and right margins. In Mozilla, the text is only left-justified.

Reproducibility

How often can you reproduce the problem?

Steps to Reproduce

Describe how to reproduce the problem, step by step. Include any special setup steps.

Actual Results

What happened after you performed the steps above?

Expected Results

What should Mozilla have done instead?

Additional Information

Add any additional information you feel may be relevant to this bug, such as the theme you were using (does the bug still occur with the default theme?), a Talkback crash ID, or special information about your computer's configuration. Any information longer than a few lines, such as a stack trace or HTML testcase, should be added using the "Create a new Attachment" link on the bug, after it is filed.

If you are reporting a crash, note the module in which Mozilla crashed (e.g., Application Violation in gkhtml.dll). On Macintosh, if you have MacsBug, attach the results of the how and sc commands.

Security

Check this box if this is a security problem that needs to be kept confidential (see the mozilla.org security policy for more details).

Severity

How serious the problem is. Also, if your bug is a request for a new feature or enhancement, say so here.

Search Form

Search form

Search Results

Bug #
Dupe
Count
Change in last
7 day(s)
Component
Severity
Op Sys
Target
Milestone
Summary
22274
126
0
 Layout
normal
All
---
 [INLINE] Too much space (for descenders) under image elements
20618
98
0
 Browser-General
major
Windows 98
---
 Mousewheel doesn't work with some driver/Win32 OS combinations (scroll, scrolling)
195600
88
0
 Installer
critical
Windows 2000
---
 browser crashes on startup after install and after xpi install. Installation of 1.4 fails
25537
83
0
 Layout
enhancement
All
---
 alt text is not displayed as a tooltip
38488
71
0
 Networking: HTTP
major
All
---
 Proxy:junkbuster is broken - use http/1.0 to get arround this
28586
62
0
 Embedding: Docshell
enhancement
All
Future
 meta bug - show error pages instead of dialogs for network errors (placeholder page in the content area) (http error pages)
11459
62
0
 Preferences
normal
All
---
 need option for mailto: to launch external mail app or open a webmail url
105547
62
0
 Tabbed Browser
normal
All
---
 Windows open in new window instead of tabs (target=<nonexistant_frame>)
69938
59
0
 File Handling
critical
All
Future
 Downloads are stored in $TMPDIR|$TMP|$TEMP|/tmp first and moved to the selected path after the download finishes
94035
58
1
 Plug-ins
enhancement
All
Future
 Allow blocking of any media type (flash, plug-in, applet, etc.) by site (like cookies and images)
193638
58
0
 Preferences: Backend
normal
All
---
 corrupt or lost pref.js / startup configuration error
167663
55
0
 XP Apps
critical
MacOS X
mozilla1.0.2
 [OS X]window disappears when clicking the maximize button (green +)
104532
53
1
 Tabbed Browser
normal
All
---
 Status bar ticker fails to update when tabs switched.

Customer Support

The steps:
  1. Greeting
  2. Learn about the problem
  3. Reproduce the problem and agree with the customer
  4. Try the problem
  5. Locate problem resolution proposals
  6. Select solution
  7. Execute solution
  8. Verify solution
  9. Agree with customer that problem has been solved
The problem life cycle
The problem life cycle

Organising the Data Centre

Data center />
Data center

Seen outside a data centre: 
                        ACHTUNG!!!

Das machine is nicht fur gefingerpoken und mittengrabben.  Ist easy
schnappen der springenwerk, blowenfusen und corkenpoppen mit
spitzensparken.  Ist nicht fur gewerken by das dummkopfen.  Das
rubbernecken sightseeren keepen hands in das pockets.  Relaxen und
vatch das blinkenlights!!!

Network Management

Possible network topologies:

Intermediate distribution frame

main distribution frame (MDF
In large buildings a main distribution frame (MDF), often located at the data centre, acts as a central hub.

Basic principles:

Distribution frame
Distribution frame. On the bottom one can see the UPS units.

Email Services

Functional units Basic operational principles:

Backups

Backup copies are required when: To optimise the backup process we can distinguish the following types of backup: Every level contains differences from previous level.

Ensure that:

References

Discussion Topics

  1. Outline the services offered by your organisation's (or the University's) data centre
  2. Provide an example where the symptom, rather than the cause is corrected
  3. Describe the naming policy and the management procedures for the .gr domain

Software development

The Software Development Process

A software process defines the following practices: of software-intensive products and services.

It can be divided into three phases:

  1. Project definition (what?)
  2. Implementation (how?)
  3. Support (change)
A process also defines a number of cross-cutting areas:

Process Maturity

CMU's Software Engineering Institute has defined the following process maturity levels:
  1. Initial
  2. Repeatable
  3. Defined
  4. Managed
  5. Optimising
Every level encompasses the elements of the previous one.

Software Process Models

Some well-known process models include the following:

Requirement Types

Product / process

Product Requirements

Levels of Detail

Requirement Drivers

Main specification drivers are: Specifications are also affected by:

Elicitation Techniques

Elicitation of nuclear power plant requirements

Non-functional Requirements

Product requirements

Process requirements

External requirements

Non-functional Requirement Metrics

Requirement Management

Advantages of Prototype Development

Prototypes help in In research examining 39 projects involving prototype development managers gave as advantages:

Prototype Development Techniques

The Visual Basic programming environment
The Visual Basic programming environment

Implementation Guidelines

(after Kernighan and Plauger 1976, Davis 1995)

Development Tools

The vim editor
The vim editor

An application wizard
An application wizard

A debugger environment
A debugger environment

The Java class documentation
The Java class documentation

Domain-specific Languages

Reuse Methods

Reuse: the Pros and Cons

Pros

Cons

Optimisation

Optimise only when: Common techniques:

Database Optimisation Techniques

Portability

Portability is hindered by: Porability can be enhanced by:

Internationalisation and Localisation

Internationalisation
Making software usable in different locales
Localisation
Adjusting the software for a given locale
Common elements of concern:

Principles of Testing

Automated testing using the JUnit framework
Automated testing using the JUnit framework

Maintenance Types and Strategies

Research has shown that maintenance effort is roughly divided into: Effort on software system can involve:
Maintenance
Fixes and new functionality
Refactoring
Architectural changes without important functional changes
Re-engineering
Re-implementation often without new architectural or functional changes

Configuration Management

Configuration management addresses the following problems: Under configuration management we typically put:

Project Management Pitfalls

Software is Different

The most important pitfalls encountered in a software development effort are (Boehm 1991):

Project Management

Project management involves addressing four elements:

Project Staffing Case Study

NuMega (Sullivan 2001) organises product development around two teams:
  1. Main team; responsible for
  2. Support team; responsible for
To a project manager report the following: The project manager is responsible for: The development manager has a reporting manager for every major functional requirement. The development manager is responsible for: A manager for each major functional requirement oversees programmers and is responsible for:

Cost Models

10 Productivity Aphorisms

(IEEE Software, Vol 4, No 5, Sept 1987, pp 84-85).

Agile Development

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: That is, while there is value in the items on the right, we value the items on the left more.
(From the Agile Manifesto web site (http://agilemanifesto.org/))

eXtreme Programming

Extreme programming is an agile development methodology valuing It involves the following practices:

Bibliography

Creative Commons License  Last change: Tuesday, January 17, 2006 4:57 pm
 Unless otherwise expressly stated, all original material on this page created by Diomidis Spinellis is licensed under a Creative Commons Attribution-Share Alike 3.0 Greece License. Portions of the above text appear in Web Services Architecture (working draft). Copyright © 2003 World Wide Web Consortium, (Massachusetts Institute of Technology, European Research Consortium for Informatics and Mathematics, Keio University). All Rights Reserved. http://www.w3.org/Consortium/Legal/2002/copyright-documents-20021231